System for vehicle display image warping

ABSTRACT

According to an aspect of the present disclosure, there is provided a system for vehicle display image warping including an occupant monitoring unit configured to check occupant monitoring information; a control command generation unit configured to generate a control command for warping an image according to the result of monitoring the occupant; and a warped image display unit configured to display a warped image according to the control command.

CROSS-REFERENCE TO RELATED APPLICATION

This application claims priority to and the benefit of Korean PatentApplication Nos. 10-2022-0004317, 10-2022-0004318, 10-2022-0004319,10-2022-0004320, 10-2022-0004321, 10-2022-0004322, 10-2022-0004323,10-2022-0004324, 10-2022-0004325, 10-2022-0004326, 10-2022-0004327,10-2022-0004328, 10-2022-0004329, and 10-2022-0004330 filed on Jan. 11,2022, the disclosure of which is incorporated herein by reference in itsentirety.

BACKGROUND 1. Technical Field

The present disclosure relates to a system for vehicle display imagewarping.

2. Related Art

Techniques for viewing an image or checking AR information using avehicle display are known.

However, when an occupant looks at an image of an inclined area, theimage is distorted according to the angle between the eyes and thedisplay.

SUMMARY

According to an aspect of the present disclosure, there is provided asystem for vehicle display image warping comprising: an occupantmonitoring unit configured to check occupant monitoring information; acontrol command generation unit configured to generate a control commandfor warping an image according to the result of monitoring the occupant;and a warped image display unit configured to display a warped imageaccording to the control command.

In the system for vehicle display image warping, the occupant monitoringunit checks at least one of a position of the user who rides in thevehicle, information on a separation distance between a reference pointof the vehicle display and the user, and gazing position information ofthe occupant.

In the system for vehicle display image warping, the control commandgeneration unit generates the control command for determining the degreeof warping according to the result of monitoring the occupant.

According to an aspect of the present disclosure, there is provided amethod for vehicle display image warping comprising: (a) monitoring anoccupant; (b) generating a control command for warping an imageaccording to a result of monitoring of the occupant; and (c) displayinga warped image according to the control command.

The step (a) checks at least one of the riding position of the occupantwho rides in the vehicle, the information on the separation distance ofthe user to the reference point of the vehicle display, or the gazingposition information of the occupant, and step (b) generates the controlcommand for determining the degree of the warping according to a resultof the checking in the step (a).

The step (c) displays the warped image, in which the height of one endof the warped image that is close to the occupant is set to be smallerthan the height of the other end of the warped image that is relativelyremote from the occupant.

According to the present disclosure, it is possible to providing avisual effect as if the occupant is viewing the image frontally withoutdistortion by performing image warping in consideration of the positionand gaze information of the occupant.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a view showing a system for controlling a vehicle based on anoccupant's intent according to an embodiment of the present disclosure.

FIGS. 2 to 4 are views showing an indoor layout change based on anoccupant's intent according to the embodiment of the present disclosure.

FIG. 5 is a view showing a method for controlling a vehicle based on anoccupant's intent according to an embodiment of the present disclosure.

FIG. 6 is a view showing a vehicle display control system based on anoccupant's gaze according to another embodiment of the presentdisclosure.

FIG. 7A and FIG. 7B are views showing vehicle windshield display controlfor each area, in consideration of priority, according to anotherembodiment of the present disclosure.

FIG. 8A and FIG. 8B are views showing the vehicle windshield displaycontrol for each area, in consideration of priority during a relativemovement of an external interest object, according to another embodimentof the present disclosure.

FIG. 9 is a view showing a vehicle display control method based on anoccupant's gaze according to another embodiment of the presentdisclosure.

FIG. 10 is a view showing an external interest information display areadetermination and vehicle display control system based on an occupant'sgaze according to another embodiment of the present disclosure.

FIG. 11A to FIG. 11C are views showing external interest informationdisplay area determination and display control according to anotherembodiment of the present disclosure.

FIG. 12 is a view showing an external interest information display areadetermination and vehicle display control method based on an occupant'sgaze according to another embodiment of the present disclosure.

FIG. 13 is a view showing an occupant gaze departure determination andvehicle display control system according to another embodiment of thepresent disclosure.

FIGS. 14 to 17 are views showing an example of vehicle display controlin an occupant gaze departure situation according to another embodimentof the present disclosure.

FIG. 18 is a view showing an example of vehicle display control upondeparture from control authority according to another embodiment of thepresent disclosure.

FIG. 19 is a view showing an occupant gaze departure determination andvehicle display control method according to another embodiment of thepresent disclosure.

FIG. 20 is a view showing a button display control in an occupantgaze-based vehicle display and initiative control system according toanother embodiment of the present disclosure.

FIGS. 21 to 23 are views showing button display control according toanother embodiment of the present disclosure.

FIG. 24 is a view showing an initiative switch in another occupant'sparticipation according to another embodiment of the present disclosure.

FIG. 25 is a view showing display control based on screen area divisionand gaze information upon another occupant's participation according toanother embodiment of the present disclosure.

FIG. 26 is a view showing a method for button display control in anoccupant gaze-based vehicle display and initiative control according toanother embodiment of the present disclosure.

FIG. 27 is a view showing a vehicle display control system fortransmitting external interest information according to anotherembodiment of the present disclosure.

FIG. 28A and FIG. 28B are views showing external interest informationdisplay and external interest information display area control accordingto another embodiment of the present disclosure.

FIG. 29 is a view showing a vehicle display control method for conveyingexternal interest information according to another embodiment of thepresent disclosure.

FIG. 30 is a view showing a vehicle display control system forsynthesizing and displaying external interest information according toanother embodiment of the present disclosure.

FIGS. 31A, 31B and 32 are views showing external interest informationdisplay and rotation according to still another embodiment of thepresent disclosure.

FIG. 33 is a view showing a vehicle display control method forsynthesizing and displaying external interest information according toanother embodiment of the present disclosure.

FIG. 34 is a view showing a vehicle display control system for storingand sharing external interest information according to anotherembodiment of the present disclosure.

FIGS. 35A through 35C are views showing a process of capturing theexternal interest information according to another embodiment of thepresent disclosure.

FIG. 36A and FIG. 36B are views showing a process of sharing theexternal interest information with a smartphone according to anotherembodiment of the present disclosure.

FIG. 37A to FIG. 37C are views showing a process of storing andconfirming external interest information for each occupant through atransparent display sliding according to another embodiment of thepresent disclosure.

FIG. 38A to FIG. 38C are views showing the sharing of the externalinterest information between displays according to another embodiment ofthe present disclosure.

FIG. 39 is a view showing a vehicle display control method for storingand sharing external interest information according to anotherembodiment of the present disclosure.

FIG. 40 is a view showing a vehicle display area control system forexternal interest information according to another embodiment of thepresent disclosure.

FIG. 41A to FIG. 41C are views showing a process of controlling avehicle display area for the external interest information according tostill another embodiment of the present disclosure.

FIG. 42 is a view showing a method of controlling a vehicle display areafor external interest information according to another embodiment of thepresent disclosure.

FIG. 43 is a view showing a vehicle display control system that displaysand stores the external interest information according to still anotherembodiment of the present disclosure.

FIG. 44A and FIG. 44B are views showing an indication of the externalinterest information according to still another embodiment of thepresent disclosure.

FIG. 45 is a view showing a vehicle display control method fordisplaying and storing the external interest information according tostill another embodiment of the present disclosure.

FIG. 46 is a block diagram illustrating a system for controlling avehicle display according to still another embodiment of the presentdisclosure.

FIG. 47 is a view illustrating a vehicle display control processaccording to the present embodiment of the present disclosure.

FIG. 48 is a flowchart illustrating a method of controlling a vehicledisplay according to still another embodiment of the present disclosure.

FIG. 49 is a block diagram illustrating a system for controlling avehicle display according to still another embodiment of the presentdisclosure.

FIGS. 50A and 50B are views illustrating that size information andvolume information of the object of interest outside of the vehicleaccording to the present embodiment of the present disclosure aredisplayed in a converted manner.

FIG. 51 is a flowchart illustrating a method of controlling a vehicledisplay according to still another embodiment of the present disclosure.

FIG. 52 is a block diagram illustrating a system for controlling avehicle display according to still another embodiment of the presentdisclosure.

FIGS. 53A and 53B are views illustrating a display presentation regionand an example of seat control when moving content according to thepresent embodiment of the present disclosure.

FIGS. 54A and 54B are views illustrating an example of control of adisplay presentation region when moving the content according to thepresent embodiment of the present disclosure.

FIG. 55 is a flowchart illustrating a method of controlling a vehicledisplay according to still another embodiment of the present disclosure.

FIG. 56 is a block diagram illustrating a system for controlling avehicle display according to still another embodiment of the presentdisclosure.

FIGS. 57A to 57C are views illustrating an example of display occupationcontrol in accordance with a result of gaze monitoring according to thepresent embodiment of the present disclosure.

FIGS. 58A and 58B are views illustrating a situation of control forrejection of the request for the initiative that is in accordance withthe result of the gaze monitoring according to the present embodiment ofthe present disclosure.

FIGS. 59A to 59C are views illustrating a situation of control for therequest for and the acceptance of the initiative that are in accordancewith the result of the gaze monitoring according to the presentembodiment of the present disclosure.

FIG. 60 is a flowchart illustrating a method of controlling a vehicledisplay according to still another embodiment of the present disclosure.

FIG. 61 is a block diagram illustrating a system for controlling avehicle display according to still another embodiment of the presentdisclosure.

FIGS. 62A and 62B are views illustrating a process of operating a screenaccording to recognition of a plurality of users according to thepresent embodiment of the present disclosure.

FIG. 63 is a view illustrating a process of personalization operationthrough display separation according to the present embodiment of thepresent disclosure.

FIGS. 64A and 64B are views illustrating a process in which a user at aremote location operates the screen according to the present embodimentof the present disclosure.

FIG. 65 is a view illustrating a process of moving an individual widgetusing a touch input signal according to the present embodiment of thepresent disclosure.

FIGS. 66A and 66B are views illustrating a process of moving theindividual widget using gaze recognition and a gesture according to thepresent embodiment of the present disclosure.

FIGS. 67A and 67B are views illustrating a process of moving a widgetgroup by moving a single widget according to the present embodiment ofthe present disclosure.

FIG. 68 is a flowchart illustrating a method of controlling a vehicledisplay according to still another embodiment of the present disclosure.

FIG. 69 is a block diagram illustrating a system for controllingreproduction of content on a medium according to still anotherembodiment of the present disclosure.

FIG. 70 is a view illustrating a process of distinguishing andrecognizing operations for content on a medium that use smartphones ofoccupants inside of and outside of the vehicle according to the presentembodiment of the present disclosure.

FIG. 71 is a view illustrating a process of distinguishing andrecognizing the operations for the content on the medium using gazerecognition information and information for the operations for thecontent on the medium that use the smartphones according to the presentembodiment of the present disclosure.

FIG. 72 is a view illustrating a process of distinguishing andrecognizing the operations for the content on the medium using the gazerecognition information and UWB communication according to the presentembodiment of the present disclosure.

FIG. 73 is a flowchart illustrating a method of controlling reproductionof content on a medium according to still another embodiment of thepresent disclosure.

FIG. 74 is a block diagram illustrating a system for controllingreproduction of content on a medium according to still anotherembodiment of the present disclosure.

FIG. 75 is a view illustrating a process of preventing soundinterference according to the present embodiment of the presentdisclosure.

FIG. 76 is a flowchart illustrating a method of controlling reproductionof content on a medium according to still another embodiment of thepresent disclosure.

FIG. 77 is a block diagram illustrating a system for determining a riskelement and providing a warning according to still another embodiment ofthe present disclosure.

FIGS. 78A to 78C are views illustrating a situation of providing awarning of an external risk element and a warning of a movement out of asafety area according to the present embodiment of the presentdisclosure.

FIG. 79 is a view illustrating a method of determining a risk elementand providing a warning according to still another embodiment of thepresent disclosure.

FIG. 80 is a block diagram illustrating a vehicle control system forreducing motion sickness according to still another embodiment of thepresent disclosure.

FIGS. 81A to 81C are views illustrating that a display and an in-vehiclelayout are changed for motion-sickness reduction according to thepresent embodiment of the present disclosure.

FIG. 82 is a view illustrating a variable display layout of a front-sidedisplay according to the present embodiment of the present disclosure.

FIGS. 83A and 83B are views illustrating a projection display (afront-side glass pane) according to the present embodiment of thepresent disclosure.

FIG. 84 is a view illustrating a projector in a situation where theprojection display (the front-side glass pane) according to the presentembodiment of the present disclosure is used.

FIGS. 85A and 85B are views illustrating the projection display (a roofglass pane) according to the present embodiment of the presentdisclosure.

FIG. 86 is a view illustrating the projector in a situation where theprojection display (the roof glass pane) according to the presentembodiment of the present disclosure is used.

FIGS. 87A to 87G are views illustrating a process of moving an imageaccording to the present embodiment of the present disclosure.

FIG. 88 is a flowchart illustrating a vehicle control method forreducing motion-sickness according to still another embodiment of thepresent disclosure.

FIG. 89 is a block diagram illustrating a system for controlling avehicle display according to still another embodiment of the presentdisclosure.

FIGS. 90A to 90D are views illustrating a process of changing a soundand an in-vehicle layout according to movement of the image betweendisplays according to the present embodiment of the present disclosure.

FIG. 91 is a flowchart illustrating a method of controlling a vehicledisplay according to still another embodiment of the present disclosure.

FIG. 92 is a block diagram illustrating a system for controlling avehicle display according to still another embodiment of the presentdisclosure.

FIGS. 93A to 93E are views illustrating a sliding display according tothe present embodiment of the present disclosure.

FIGS. 94A to 94C are views illustrating a process of moving the displayand displaying AR content according to the present embodiment of thepresent disclosure.

FIG. 95 is a flowchart illustrating a method of controlling a vehicledisplay according to still another embodiment of the present disclosure.

FIG. 96 is a block diagram illustrating a system for warping an image ona vehicle display according to still another embodiment of the presentdisclosure.

FIGS. 97A and 97B are views illustrating an example of an image based oneach occupant's position and an example of warping according to thepresent embodiment of the present disclosure.

FIG. 98 is a view illustrating the warping of the image according togaze information of the occupant according to the present embodiment ofthe present disclosure.

FIG. 99 is a view illustrating the warping of the image according to adistance to the occupant and the gaze information according to thepresent embodiment of the present disclosure.

FIG. 100 is a view illustrating a method of warping an image on avehicle display according to still another embodiment of the presentdisclosure.

FIG. 101 is a block diagram illustrating a system for controlling avehicle display according to still another embodiment of the presentdisclosure.

FIGS. 102A and 102B are views illustrating that the display is dividedinto regions according to the distance to the occupant according to thepresent embodiment of the present disclosure.

FIG. 103 is a flowchart illustrating a method of controlling a vehicledisplay according to still another embodiment of the present disclosure.

FIG. 104 is a view illustrating a purpose-built vehicle according tostill another embodiment of the present disclosure.

FIG. 105 is a vertical cross-sectional view taken along line A-A on FIG.

FIGS. 106 and 107 are horizontal cross-sectional views taken along lineA-A on FIG. 104 .

FIGS. 108 to 114 and FIG. 115 are views illustrating various seatstructures, respectively, of the purpose-built vehicle according to thepresent embodiment of the present disclosure.

DETAILED DESCRIPTION

The following detailed description is provided to assist the reader ingaining a comprehensive understanding of the methods, apparatuses,and/or systems described herein. However, various changes,modifications, and equivalents of the methods, apparatuses, and/orsystems described herein will be apparent after an understanding of thedisclosure of this application. For example, the sequences of operationsdescribed herein are merely examples, and are not limited to those setforth herein, but may be changed as will be apparent after anunderstanding of the disclosure of this application, with the exceptionof operations necessarily occurring in a certain order.

The features described herein may be embodied in different forms and arenot to be construed as being limited to the examples described herein.Rather, the examples described herein have been provided merely toillustrate some of the many possible ways of implementing the methods,apparatuses, and/or systems described herein that will be apparent afteran understanding of the disclosure of this application.

Advantages and features of the present disclosure and methods ofachieving the advantages and features will be clear with reference toembodiments described in detail below together with the accompanyingdrawings. However, the present disclosure is not limited to theembodiments disclosed herein but will be implemented in various forms.The embodiments of the present disclosure are provided so that thepresent disclosure is completely disclosed, and a person with ordinaryskill in the art can fully understand the scope of the presentdisclosure. The present disclosure will be defined only by the scope ofthe appended claims. Meanwhile, the terms used in the presentspecification are for explaining the embodiments, not for limiting thepresent disclosure.

Terms, such as first, second, A, B, (a), (b) or the like, may be usedherein to describe components. Each of these terminologies is not usedto define an essence, order or sequence of a corresponding component butused merely to distinguish the corresponding component from othercomponent(s). For example, a first component may be referred to as asecond component, and similarly the second component may also bereferred to as the first component.

Throughout the specification, when a component is described as being“connected to,” or “coupled to” another component, it may be directly“connected to,” or “coupled to” the other component, or there may be oneor more other components intervening therebetween. In contrast, when anelement is described as being “directly connected to,” or “directlycoupled to” another element, there can be no other elements interveningtherebetween.

The singular forms “a”, “an”, and “the” are intended to include theplural forms as well, unless the context clearly indicates otherwise. Itwill be further understood that the terms “comprises/comprising” and/or“includes/including” when used herein, specify the presence of statedfeatures, integers, steps, operations, elements, and/or components, butdo not preclude the presence or addition of one or more other features,integers, steps, operations, elements, components and/or groups thereof.

FIG. 1 is a view showing a system for controlling a vehicle based on anoccupant's intent according to an embodiment of the present disclosure.

The system for controlling a vehicle based on an occupant's intentaccording to the embodiment of the present disclosure includes: anoccupant gaze recognition unit 101 configured to recognize a gaze of anoccupant in a vehicle; a point of interest determination unit 103configured to determine a point of interest, among external objectsviewed through a side window of the vehicle in consideration of the gazeof the occupant; and a seat position control unit 105 configured tochange a seat position of the occupant to enable a continuous stare atthe point of interest as the vehicle moves.

The system for controlling a vehicle based on an occupant's intentaccording to the present disclosure further includes: an occupantgesture recognition unit 102 configured to recognize at least any one ofa touch input signal of the occupant and an air gesture of the occupantthat is applied to the side window of the vehicle configured as atransparent display; and the point of interest determination unit 103determines the point of interest as at least any one of the touch inputsignal and the air gesture is recognized.

The seat position control unit 105 transmits a control signal to rotatea seat in which the occupant is seated, in consideration of a travelingdirection and travel speed of the vehicle.

The seat position control unit 105 transmits a control signal to rotatea seat of a first occupant, in consideration of gazes of a plurality ofvehicle occupants that stare the same point of interest, and transmits acontrol signal to rotate a seat of a second occupant with forward orbackward sliding for the seat of the second occupant.

The system for controlling a vehicle based on an occupant's intentaccording to the present disclosure further includes a display controlunit 104 configured to control, when it is confirmed that the gazes ofthe plurality of vehicle occupants stare at the same point of interestand then it is determined that a field of view of the second occupant isblocked by the first occupant, an outer area, viewed through a firstside window close to a seating position of the first occupant, to bedisplayed through a second side window close to a seating position ofthe second occupant. At this time, the seat position control unit 105transmits the control signal to rotate the seat of the first occupant ina first direction, and transmits the control signal to rotate the seatof the second occupant in an opposite direction to the first direction.

FIGS. 2 to 4 are a view showing an indoor layout change based on anoccupant's intent according to the embodiment of the present disclosure.

Referring to FIGS. 2 to 4 , it is assumed that among persons in thevehicle traveling forward, there is a first occupant in a left rear seatand a second occupant in a right rear seat.

The first occupant is a person close to a transparent display of a leftside window, and performs touch operations on the transparent display,and the second occupant is a person far from the transparent display ofthe left side window, and performs an operation through an air gesture.

According to the embodiment of the present disclosure, the gaze of theoccupant is recognized and augmented reality (AR) informationpersonalized for each occupant based on a gaze information is displayedthrough the transparent display of the left side window.

Referring to FIG. 2 , when the first occupant inputs a touch signal intothe area of an external object (isolated spire) viewed through thetransparent display of the left side window, the point of interestdetermination unit 103 determines that an external particular object isthe interest object, in consideration of the gaze of the first occupantand the touch signal application area of the first occupant.

The display control unit 104 displays specific information about theinterest object through the transparent display of the left side window,in the form of text, images, icons, etc.

In addition, the display control unit 104 may provide the informationabout the interest object as a sound source through a speaker and mayalso independently provide the sound source to the first occupantthrough a directional speaker or a headrest speaker disposed at the leftrear seat.

As the vehicle travels forward, the external interest object viewedthrough the left side window is moved in a left backward directionrelative to a vehicle body.

When it is checked that the gaze of the first occupant stays on theinterest object and the first occupant turns his/her head in the leftbackward direction and intends to continuously stare at the interestobject, the seat position control unit 105 rotates the left rear seat,on which the first occupant is seated, in a counterclockwise direction,allowing the first occupant to continuously stare at the interest objectin a more comfortable position.

At this time, the seat position control unit 105 determines that thefirst occupant wants to continuously stare at the interest object when astaring duration time of the first occupant for the interest objectobtained through the occupant gaze recognition unit 101 is equal to orlonger than a preset time, and the seat position control unit 105controls a rotational speed of the left rear seat on which the firstoccupant is seated in accordance with the driving speed of the drivingvehicle and the relative movement speed of the external interest object.

When the seat position control unit 105 predicts that a sharp seatrotation will be required and will cause discomfort to the firstoccupant in a case in which the seat is rotated in accordance with therelative movement of the external interest object during high-speeddriving equal to or higher than a predetermine speed (e.g., 50 Km/h),the seat position control unit 105 does not perform the rotationoperation on the seat.

The occurrence of discomfort is predicted based on a predeterminedreference seat rotation speed.

At this time, the seat position control unit 105 determines whether toperform a seat rotation operation based not only on the driving speed ofthe driving vehicle, but also on a distance information between theexternal interest object and the vehicle.

For example, in a situation where an external interest object is 50meters away from the driving vehicle, when the seat is rotated so thatthe first occupant can continuously stare at the external interestobject at the driving speed, a sharp seat rotation is required, whereaswhen an external interest object is 500 meters away from the drivingvehicle, even if the seat is rotated so that the first occupant cancontinuously stare at the external interest object at the driving speed,a sharp seat rotation is not required.

Thus, the seat position control unit 105 comprehensively considers thedriving speed of the driving vehicle and the distance between theexternal interest object at which the first occupant wants tocontinuously stare and the driving vehicle, to determine whether torotate the seat of the first occupant so that the first occupant cancontinuously and comfortably stare at the external interest objectwithout turning his/her head.

As the vehicle travels further forward, when it is determined that theinterest object is out of the range of vision of the first occupant, theseat position control unit 105 rotates the left rear seat, on which thefirst occupant is seated, in a clockwise direction to return the leftrear seat to its original position.

In addition, the seat position control unit 105 rotates the left rearseat, on which the first occupant is seated, in the clockwise directionto return the left rear seat to its original position when the gazeinformation obtained through the occupant gaze recognition unit 101 nolonger stays on the interest object for a certain period of time orlonger.

Referring to FIG. 3 , it is assumed that the first occupant inputs atouch signal into a corresponding area while staring at an externalobject (isolated spire) viewed through the transparent display of theleft side window, and the second occupant performs a predetermined airgesture while staring at the same external object (isolated spire)viewed through the transparent display of the left side window.

That is, the first occupant and the second occupant designate the sameexternal object as an external point of interest (POI).

The point of interest determination unit 103 and determines that anexternal particular object is the interest object, in consideration ofthe gaze of the first occupant, the touch signal application area of thefirst occupant, the gaze of the second occupant, and the air gesture ofthe second occupant.

The display control unit 104 displays a specific information of theinterest object through the transparent display of the left side window,in the form of text, images, icons, etc.

In addition, the display control unit 104 may provide the informationabout the interest object as a sound source through a speaker, and inthis case, the display control unit 104 may also provide the soundsource to the first and second occupants, through a directional speakerfacing the occupants seated on the rear seats, a headrest speakerdisposed at the left rear seat, and a headrest speaker disposed at theright rear seat.

As the vehicle travels forward, the external interest object viewedthrough the left side window is moved in the left backward directionrelative to the vehicle body.

When it is confirmed that the gazes of the first occupant and the secondoccupant stay on the interest object and the first occupant and thesecond occupant are then continuously staring at the interest object byturning their heads in the left backward direction, the seat positioncontrol unit 105 rotates the left rear seat, on which the first occupantis seated, in the counterclockwise direction, allowing the firstoccupant to continuously stare at the interest object in a morecomfortable position.

In addition, in consideration of the field of view of the secondoccupant obscured by the first occupant as the external interest objectis relatively moved, the seat position control unit 105 slides the rightrear seat, on which the second occupant is seated, backward of thevehicle body and rotates the right rear seat in the counterclockwisedirection, so that the second occupant can more easily and continuouslystare at the interest object.

As the vehicle travels further forward, when it is determined that theinterest object is out of the ranges of visions of the first occupantand the second occupant, the seat position control unit 105 rotates theleft rear seat, on which the first occupant is seated, in the clockwisedirection, and slides forward the right rear seat on which the secondoccupant is seated and rotates the right rear seat in the clockwisedirection, to return the left and right rear seats to their originalseat positions.

Referring to FIG. 4 , it is assumed that the first occupant inputs atouch signal into the area while staring at the external object(isolated spire) viewed through the transparent display of the left sidewindow, and the second occupant performs a predetermined air gesturewhile staring at the same external object (isolated spire) viewedthrough the transparent display of the left side window.

That is, the first occupant and the second occupant designate the sameexternal object as the external POI.

The point of interest determination unit 103 determines that an externalparticular object is the interest object, in consideration of the gazeof the first occupant, the touch signal application area of the firstoccupant, the gaze of the second occupant, and the air gesture of thesecond occupant.

The display control unit 104 displays the specific information of theinterest object through the transparent display of the left side window,in the form of text, images, icons, etc.

In addition, the display control unit 104 may provide the informationabout the interest object as a sound source through a speaker, and inthis case, the display control unit 104 may also provide the soundsource to the first and second occupants, through a directional speakerfacing the occupants seated on the rear seats, a headrest speakerdisposed at the left rear seat, and a headrest speaker disposed at theright rear seat.

As the vehicle travels forward, the external interest object shownthrough the left side window is moved in a left backward directionrelative to the vehicle body.

When it is confirmed that the gaze of the first occupant stays on theinterest object and the first occupant is then continuously staring atthe interest object by turning his/her head in the left backwarddirection, the seat position control unit 105 rotates the left rearseat, on which the first occupant is seated, in the counterclockwisedirection, allowing the first occupant to continuously stare at theinterest object in a more comfortable position.

In addition, in consideration of the field of view of the secondoccupant blocked by the first occupant as the external interest objectis relatively moved, the display control unit 104 duplicates anddisplays the screen of an outer landscape viewed through the transparentdisplay of the left side window through the display of the right sidewindow, and the seat position control unit 105 rotates the right rearseat, on which the second occupant is seated, in the clockwise directionso that the second occupant can more easily and continuously stare atthe interest object through the display of the right side window.

When it is determined that the interest object is out of the ranges ofviews of the first occupant and the second occupant as the vehicletravels further forward, the seat position control unit 105 rotates theleft rear seat, on which the first occupant is seated, in the clockwisedirection, and rotates the right rear seat, on which the second occupantis seated, in the counterclockwise direction, to return the left andright rear seats to their original seat positions.

FIG. 5 is a view showing a method of controlling a vehicle based on anoccupant's intent according to another embodiment of the presentdisclosure.

The method of controlling a vehicle based on an occupant's intentaccording to the embodiment of the present disclosure includes: a stepS501 for obtaining gaze information of an occupant in a vehicle; a stepS502 for determining a point of interest located outside the vehicleusing the gaze information; and a step S503 for changing a seat positionof the occupant to enable a continuous stare at the point of interest asthe vehicle moves.

In the step S503, a seat on which the occupant is seated is rotated inconsideration of the driving direction and driving speed of the vehicle.

When the gaze information of a plurality of occupants is obtained in thestep S501, and the points of interest of the plurality of occupants aredetermined to be the same in the step S502, a seat of a first occupantis rotated, and a seat of a second occupant is slid forward or backwardand rotated, in the step S503.

The method of controlling a vehicle based on an occupant's intentaccording to the present disclosure further includes a step S504 forcontrolling an outer area, including the points of interest, which areviewed through a first side window close to the seating position of thefirst occupant, to be displayed through a second side window close tothe seating position of the second occupant, when the gaze informationof the plurality of occupants is obtained in the step S501 and thepoints of interest of the plurality of occupants are determined to bethe same in the step S502.

FIG. 6 is a view showing a vehicle display control system based on anoccupant's intent according to another embodiment of the presentdisclosure.

The vehicle display control system based on an occupant's intentaccording to the embodiment of the present disclosure includes: a firstoccupant gaze recognition unit 601 configured to recognize a gaze of afirst occupant in a vehicle; a second occupant gaze recognition unit 602configured to recognize a gaze of a second occupant in the vehicle; aunit for confirming an interest information display area crossover anddetermining priority 603 configured to determine interest informationfrom the gazes of the first occupant and the second occupant, to confirmwhether areas for displaying the interest information crossover witheach other on a windshield, and to determine a priority for display; anda display control unit 604 configured to control the display accordingto the priority for display.

The display control unit 604 performs display control in considerationof a predetermined initiative information when an interest informationdisplay area of the first occupant and an interest information displayarea of the second occupant overlaps with each other.

When the first occupant, not the second occupant, is set to have theinitiative, The display control unit 604 performs control to display theinterest information display area of the first occupant at a topposition while covering the interest information display area of thesecond occupant with the interest information display area of the firstoccupant.

When the first occupant, not the second occupant, is set to have theinitiative, the display control unit 604 performs control to move theinterest information display area of the second occupant and to allowthe interest information display area of the second occupant not tooverlap with the interest information display area of the firstoccupant.

The display control unit 604 performs control in consideration of theimportance of the interest information when the interest informationdisplay area of the first occupant and the interest information displayarea of the second occupant overlap with each other.

When the display control unit 604 determines that the interestinformation of the first occupant has the importance higher than that ofthe interest information of the second occupant, the display controlunit 604 performs control not to display the interest information of thesecond occupant, but to display the interest information of the firstoccupant.

In consideration of at least any one of the types of content displayedin a display area assigned to the first occupant and an operationsituation associated with the display area assigned to the firstoccupant, the display control unit 604 determines whether to display theinterest information of the second occupant in the display area assignedto the first occupant.

FIG. 7A and FIG. 7B are views showing vehicle windshield display controlfor each area, in consideration of priority, according to anotherembodiment of the present disclosure.

Referring to FIG. 7A, a driver A and a passenger B look at externalobjects through a front windshield of a vehicle.

At this time, when the driver A stares at a forward vehicle located inan area of □ for a preset time or longer, driving-related information isdisplayed according to a result of tracking a gaze of the driver A, forexample, a distance from the forward vehicle, navigationinformation-based AR route guidance information, etc. are displayed in afirst area A-1 of the front windshield which is a transparent display.

The passenger B stares at a store located in an area of □ for the presettime or longer, and accordingly, the information of the store (e.g.,summer sale, up to 70%) is displayed in a second area B-1 of the frontwindshield, which is the transparent display.

Referring to FIG. 7B, the driver A looks at a parking tower (e.g., theparking tower is a destination set on a navigation route) located in thearea of □, and accordingly, the information of the parking tower (e.g.,parking cost per hour) is displayed in the second area B-1 of the frontwindshield, which is the transparent display.

At this time, as described above, a display area (that is, some of thesecond area B-1 of the front windshield, which is the transparentdisplay) for the information (e.g., summer sale, up to 70%) of the storelocated in the area of □ (e.g., summer sale, up to 70%) according to thegaze of the passenger B and a display area for the information of theparking tower located in the area of □ according to the gaze of thedriver A overlap with each other.

For example, the display control unit 604 performs control to displaythe external interest information area of the driver A and the externalinterest information area of the passenger B while overlapping the areaswith each other. At this time, on the contrary, as the gaze of thepassenger B stares at the store located in the first area A-1, whendisplaying the information about the store located in the first area A-1on the front windshield, which is the transparent display, if it isdetermined that the display area for the information will overlap withthe display area for the object at which driver A stares, the displaycontrol unit 604 performs control to display only the information aboutthe object, at which the driver A stares, but not to display theexternal interest information according to the gaze of the passenger B.

As another example, the display control unit 604 confirms that thedisplay priority of the information displayed on the transparent displayaccording to the gaze of driver A is higher than the display level ofthe information displayed on the transparent display according to thegaze of the passenger B, and displays the external interest informationarea of the driver A in front of the external interest information areaof the passenger B (i.e., some of the external interest information areaof the passenger B may be covered).

As still another example, the display control unit 604 confirms that thedisplay priority of the information displayed on the transparent displayaccording to the gaze of the driver A is higher than the display levelof the information displayed on the transparent display according to thegaze of the passenger B, and moves the display area for the externalinterest information of the passenger B so that the display area doesnot overlap with the display area for the external interest informationof the driver A.

For example, in the case of FIG. 7B, the display control unit 604performs control to move the display area for the information of thestore (e.g., summer sale, up to 70%) to the bottom right, so that thedisplay area does not overlap with the display area for the informationof the parking tower.

That is, when the driver A, not the passenger B, has the initiative, thedisplay control unit 604 performs control to display the display areafor the external interest information, at which the driver A stares, inpreference to the display area for the external interest information atwhich the passenger B stares.

In addition, when the information provided according to the gazerecognition of the occupant is driving, safety-related information(e.g., a lane, a speed limit, a traffic light, information about adistance from a forward vehicle, etc.), the display control unit 604performs control to display a display area for the external interestinformation in preference to other information (e.g., store information,etc.) display area.

FIG. 8A and FIG. 8B are views showing the vehicle windshield displaycontrol for each area, in consideration of priority during a relativemovement of an external interest object, according to the embodiment ofthe present disclosure.

Referring to FIG. 8A, a first occupant A and a second occupant B look anexternal object through a windshield of a left side window of a vehicle.

At this time, a music player is displayed in a first area A-1 accordingto the needs of the first occupant A, and an outline of the externalinterest object (Isolated Spire) is displayed in a second area B-1according to the needs of the second occupant B.

Referring to FIG. 8B, as the vehicle travels forward, when the interestinformation of the second occupant B invades a display area A-1 of thefirst occupant A, the display control unit 604 performs display controlfor the first area A-1 and the second area B-1 in consideration of theattributes of the content and the gaze monitoring results of the firstoccupant A.

Referring to FIG. 8B, when the content displayed in the display area A-1of the first occupant A is a content (e.g., music, weather, movies,etc.) that is not the external interest information based on the gazeinformation of the first occupant A, depending on the type of content orthe screen operation state of the first occupant A, the display controlunit 604 determines whether to display the information of the externalinterest object of the second occupant B on the first area A-1 while theinformation invades the first area A-1.

When the first occupant A selects music in the first area A-1, thedisplay control unit 604 performs control not to display the informationof the external interest object of the second occupant B in the firstarea A-1 so that the information does not interfere with the screenoperation of the first occupant A.

In addition, if the first occupant A is watching a movie in the firstarea A-1, the display control unit 604 performs control not to displaythe information of the external interest object of the second occupant Bin the first area A-1 so that the information does not interfere withthe first occupant A that watches a movie.

As another example, when the first occupant A is not performing anoperation on the content execution in the first area A-1, the displaycontrol unit 604 performs control to display the information of theexternal interest object of the second occupant B in the first area A-1.

In addition, if it is confirmed that the first occupant A is staring atanother place for a certain period of time or longer, as in □, thedisplay control unit 604 performs control to display the information ofthe external interest object of the second occupant B in the first areaA-1.

FIG. 9 is a view showing a vehicle display control method based on anoccupant's gaze according to another embodiment of the presentdisclosure.

The vehicle display control method based on an occupant's gaze accordingto the embodiment of the present disclosure includes: a step S901 forrecognizing gaze information of a plurality of occupants in a vehicle; astep S902 for confirming interest information using the gazeinformation; a step S903 for confirming whether display areas of theinterest information displayed on a windshield being a transparentdisplay crossover with each other; and a step S904 for performingdisplay control when the display areas of the interest informationcrossover.

In the step S904, the display control is performed in consideration ofpredetermined initiative information according to the result that it isconfirmed that an interest information display area of the firstoccupant and an interest information display area of the second occupantoverlap with each other.

In the step S904, when the first occupant, not the second occupant, isset to have an initiative, the display control is performed such thatthe interest information display area of the first occupant is displayedat a top position while covering the interest information display areaof the second occupant.

In step S904, when the first occupant, not the second occupant, is setto have the initiative, the display control is performed such that theinterest information display area of the second occupant is moved sothat the interest information display area of the second occupant doesnot overlap with the interest information display area of the firstoccupant.

In the step S904, when the interest information display area of thefirst occupant and the interest information display area of the secondoccupant overlap with each other, the display control is performed inconsideration of the importance of the interest information.

In the step S904, when it is determined that the interest information ofthe first occupant the importance higher than that of the interestinformation of the second occupant, the display control is performedsuch that the interest information of the second occupant is notdisplayed, but the interest information of the first occupant isdisplayed.

In the step S904, in consideration of at least any one of the types ofcontent displayed in the display area assigned to the first occupant andthe operation situation associated with the display area assigned to thefirst occupant, it is determined whether to display the interestinformation of the second occupant in the display area assigned to thefirst occupant.

FIG. 10 is a view showing an external interest information display areadetermination and vehicle display control system based on an occupant'sgaze according to another embodiment of the present disclosure.

The external interest information display area determination and vehicledisplay control system based on an occupant's gaze according to theembodiment of the present disclosure includes: a first occupant gazerecognition unit 1001 configured to recognize a gaze of a first occupantin a vehicle; an external interest information confirmation unit 1003 ofthe first occupant configured to confirm an external interestinformation of the first occupant from the gaze of the first occupant; asecond occupant gaze recognition unit 1002 configured to recognize agaze of a second occupant in the vehicle; an external interestinformation confirmation unit 1004 of the second occupant configured toconfirm the external interest information of the second occupant fromthe gaze of the second occupant; an external interest informationdisplay area confirmation unit 1005 configured to confirm a display areafor external interest information of the first occupant and a displayarea for external interest information of the second occupant; and adisplay control unit 1006 configured to determine a priority fordisplay, in consideration of a recognition influence on the display areafor the external interest information of the first occupant by thedisplay area for the external interest information of the secondoccupant with relatively low information display importance.

When it is determined that the recognition influence is equal to orlower a predetermined value, the display control unit 1006 performsdisplay control to display the external interest information of thesecond occupant together with the external interest information of thefirst occupant.

When it is determined that the recognition influence exceeds thepredetermined value, the display control unit 1006 performs displaycontrol not to display the external interest information of the secondoccupant, but to display only the external interest information of thefirst occupant.

When it is determined that the recognition influence exceeds thepredetermined value, the display control unit 1006 performs displaycontrol to move the display area for the external interest informationof the second occupant so that the moved display area for the externalinterest information of the second occupant does not invade the displayarea for the external interest information of the first occupant.

The display control unit 1006 performs display control to display anoutline information about an original position of an external interestobject of the second occupant.

FIG. 11A to FIG. 11C are views showing external interest informationdisplay area determination and display control according to anotherembodiment of the present disclosure.

Referring to FIG. 11 , a driver A and a passenger B in a vehicle look atexternal objects through a front windshield of the vehicle.

Referring to FIG. 11A, when the driver A stares at a forward vehiclelocated in the area of □ for a preset time or longer, driving-relatedinformation is displayed according to a result of tracking a gaze of thedriver A, for example, a distance from the forward vehicle, navigationinformation-based AR route guidance information, etc. are displayed in afirst area A-1 of the front windshield configured as a transparentdisplay. The passenger B stares at a store located in the area of □ forthe preset time or longer, and accordingly, the information of the store(e.g., summer sale, up to 70%) is displayed in a second area B-1 of thefront windshield, which is a transparent display.

Referring to FIG. 11B, according to the gaze movement of the passengerB, a display area for the external interest information corresponding to□ is moved to the first area A-1.

At this time, the display control unit 1006 performs control to displaythe external interest information of the passenger B in the first areaA-1 when it is confirmed that the display area for the external interestinformation of the passenger B does not overlap with the display areafor the external interest information of the driver A.

Referring to FIG. 11C, according to the gaze movement of the driver A, arear side warning notification corresponding to □ is displayed as theexternal interest information of the driver A.

In this case, it is determined that the display area for the externalinterest information of the passenger B described in FIG. 11B overlapswith the display area for the external interest information of thedriver A, and the display control unit 1006 determines whether todisplay the external interest information of the passenger B.

At this time, the display control unit 1006 monitors the gazeinformation of the driver A and does not display the external interestinformation of the passenger B until the display area for the externalinterest information of the driver A does not overlap with the displayarea for the external interest information of the passenger B.

As another example, as shown in FIG. 11C, the display control unit 1006moves the display area for the external interest information ofpassenger B so that the display area for the external interestinformation of the passenger B does not overlap with the display areafor the external interest information of the driver A.

As shown in FIG. 11C, the display control unit 1006 moves the displayarea for the external interest information of the passenger B to thesecond area B-1 so that no obstacle interferes with the driver A toconfirm the external interest information.

The display control unit 1006 performs control to display the externalinterest information of the passenger B as AR or displayed as a cameravideo.

The display control unit 1006 continuously displays the position of theexternal interest object of the passenger B in the form of a dotted lineso that the position is recognized, and thus the display control unit1006 performs control for the position display not to interfere with thegaze of the driver A.

FIG. 12 is a view showing an external interest information display areadetermination and vehicle display control method based on an occupant'sgaze according to another embodiment of the present disclosure.

The external interest information display area determination and vehicledisplay control method based on an occupant's gaze according to theembodiment of the present disclosure includes: a step S1201 forrecognizing gaze information of a first occupant and a second occupantin a vehicle; a step S1202 for confirming a display area for externalinterest information of the first occupant and a display area forexternal interest information of the second occupant using the gazeinformation of the first occupant and the second occupant; and a stepS1203 for determining a priority for display and controlling the displayin consideration of an recognition influence on the display area for theexternal interest information of the first occupant by the display areafor the external interest information of the second occupant withrelatively low information display importance.

In the step S1203, when it is determined that the recognition influenceis equal to or lower than a predetermined value, the display control isperformed such that the external interest information of the secondoccupant is displayed together with the external interest information ofthe first occupant.

In step S1203, when it is determined that the recognition influenceexceeds the predetermined value, the display control is performed suchthat the external interest information of the second occupant is notdisplayed, but only the external interest information of the firstoccupant is displayed.

In step S1203, when it is determined that the recognition influenceexceeds the predetermined value, the display control is performed suchthat the display area for the external interest information of thesecond occupant is moved, and the moved display area for the externalinterest information of the second occupant does not invade the displayarea for the external interest information of the first occupant.

In step S1203, the display control is performed such that an outlineinformation about an original position of the external interest objectof the second occupant is displayed.

FIG. 13 is a view showing an occupant gaze departure determination andvehicle display control system according to another embodiment of thepresent disclosure.

The occupant gaze departure determination and vehicle display controlsystem according to the embodiment of the present disclosure includes:an occupant information recognition unit 1310 configured to recognizebehavioral information related to viewing a display screen of anoccupant in a vehicle; and a display control unit 1320 configured toperform control over the vehicle display using the behavior informationrelated to viewing the display screen, and the display control unit 1320analyzes the behavioral information related to viewing the displayscreen to determine that a gaze of the occupant departs from a displayscreen area, and accordingly, performs display control.

The occupant information recognition unit 1310 includes an occupant gazerecognition unit 1311 configured to obtain the gaze information of theoccupant in the vehicle and recognize whether the gaze of the occupantin the vehicle departs from the display screen area, and a departureduration time.

The display control unit 1320 performs control to blur the screen of avideo content when the duration time that the gaze of the occupantdeparts from the display screen area exceeds a preset time.

The occupant information recognition unit 1310 further includes a voicerecognition unit 1312 configured to obtain in-vehicle voice information,and the display control unit 1320 confirms that a call or in-vehicleconversation is in progress, using the in-vehicle voice information, andperforms control to pause for playback of video content corresponding tovideo playback.

The display control unit 1320 confirms that the call or in-vehicleconversation is in progress, using the in-vehicle voice information, andperforms control to lower a playback volume of the video content.

The display control unit 1320 performs control to display a wallpaperrather than the video content when the departure duration time lasts forthe preset time or longer.

The occupant information recognition unit 1310 further includes asitting status recognition unit 1313 configured to recognize theoccupant's sitting status for a seat in the vehicle, and the displaycontrol unit 1320 performs control to confirm the occupant approachingthe seat to auto-play the video content when a first occupant who hasthe initiative is seated on the seat.

When it is confirmed that a second occupant who has no initiative isseated on the seat, the display control unit 1320 assigns a portion ofthe display screen area to the second occupant.

The display control unit 1320 performs control to display the externalinterest information of the second occupant beyond the area assigned tothe second occupant, in consideration of at least any one ofauthorization setting information of the first occupant and boardingstatus information of the first occupant.

When the first occupant leaves the vehicle, the display control unit1320 performs control to display a function previously used by thesecond occupant on the screen in consideration of usage historyinformation of the second occupant.

FIGS. 14 to 17 are views showing an example of vehicle display controlin an occupant gaze departure situation according to another embodimentof the present disclosure.

Referring to FIG. 14 , when a gaze of an occupant (who has theinitiative) who is watching a video content being played on a vehicledisplay deviates from a screen, the display control unit 1320 performscontrol to blur the screen of the vehicle display so that the videocontent displayed on the screen does not interfere with the occupant.

At this time, when it is determined that the occupant makes a call, ortalks to another occupant in the vehicle, or is leaning on the seat withhis eyes closed, etc., that is, when it is determined that the occupanthas no intent to watch the video content, the display control unit 1320performs control to lower the playback volume of the video content.

At this time, when the video content corresponds to video playbackrather than real-time playback, such as TV content, the display controlunit 1320 performs control to pause the video content.

Referring to FIG. 15 , when the occupant that is watching the videocontent being played on the vehicle display moves to another seat ormoves, if the gaze movement of the occupant is within a reference value(e.g., if the gaze returns back into the screen area within 3 seconds)even if the gaze of the occupant is out of a preset range (e.g.: screenarea), the display control unit 1320 performs control to play back thevideo content as it is without changing the video content screen andplayback sound.

When the occupant leaves the seat, the display control unit 1320performs control to lower the playback volume of the video content, toblur the vehicle display screen (at this time, control to lower theplayback volume decreases sequentially over time from the moment thatthe occupant leaves the seat, and to gradually blur the screen), and topauses the video content if the video content corresponds to videoplayback rather than real-time playback.

If the occupant does not return to the seat even after the preset timefrom the moment that the occupant leaves the seat, the display controlunit 1320 performs control to convert the screen on which the videocontent is played into a wallpaper to display the wallpaper so that noother occupant cannot see the screen.

Referring to FIG. 16 , when detecting that the occupant who has left theseat is approaching the seat, while the wallpaper is being displayed onthe vehicle display, the display control unit 1320 performs control tostop the wallpaper display and to display the previously paused videocontent.

At this time, when it is confirmed that the occupant seated on the seathas the initiative, the display control unit 1320 performs control toauto-play the video content.

When the occupant who has approached the seat has the initiative, but isnot seated on the seat with only the occupant's gaze located on thescreen, the occupant clicks a play button to play the video content.

Referring to FIG. 17 , as the gaze of the occupant deviates from thevehicle display screen for a certain time or longer, the wallpaper isdisplayed.

At this time, when another occupant that has no initiative approachesthe seat, the display control unit 1320 assigns a portion of thecorresponding vehicle display screen area to another occupant.

At this time, when a screen lock is set, the display control unit 1320performs control not to display the video content that was previouslyplayed in the area assigned to the occupant having the initiative.

The area provided to another occupant (occupant B) who has no initiativeis shown as a second area B-1, and the area provided to the occupant(occupant A) who has the initiative is shown as a first area A-1.

When the external interest information of the occupant B is to bedisplayed in the first area A-1 beyond the second area B-1, the displaycontrol unit 1320 performs control to display even the external interestinformation of the occupant B in the first area A-1 according to theboarding status of the occupant A or whether the authority of theoccupant A having the initiative is set.

FIG. 18 is a view showing an example of vehicle display control upondeparture from control according to another embodiment of the presentdisclosure.

In a situation where the video content is played on the vehicle display,when the occupant having the initiative leaves the vehicle, the displaycontrol unit 1320 performs display control to display the wallpaper.

When the occupant having the initiative leaves the vehicle, the displaycontrol unit 1320 performs display control to display the functionspreviously used by another occupant as another occupant having noinitiative approaches the display.

FIG. 19 is a view showing an occupant gaze departure determination andvehicle display control method according to another embodiment of thepresent disclosure.

The occupant gaze departure determination and vehicle display controlmethod according to the embodiment of the present disclosure includes: astep S1901 for obtaining at least any one of gaze information, voiceinformation, and sitting status information of an occupant in a vehicle;and a step S1902 for confirming that the occupant's intent is to stopviewing a display screen, by using the information obtained in the stepS1901, and controlling a vehicle display.

In the step S1901, when the gaze information of the occupant in thevehicle is obtained, the gaze information including whether the gaze ofthe occupant in the vehicle departs from the display screen area and thedeparture duration time is obtained.

In the step S1902, when the departure duration time exceeds the presettime, control is performed such that the screen of the video content isblurred.

When the voice information is obtained in the step S1901, in the stepS1902, it is determined that a call or an in-vehicle conversation is inprogress, using the voice information, and control is performed suchthat the video content corresponding to the video playback is paused.

When the voice information is obtained in the step S1901, in the stepS1902, it is determined that the call or in-vehicle conversation is inprogress, using the voice information, and control is performed suchthat the playback volume of the video content is lowered.

In the step S1902, it is confirmed that the occupant's intent is to stopviewing the display screen, and control is performed such that thewallpaper, not the video content, is displayed.

When the sitting status information is recognized in the step S1901, thevideo content is automatically displayed depending on whether theoccupant seated in the seat in the step S1902 has been granted theinitiative.

When it is confirmed in the step S1901 that an occupant who has beengranted no initiative is seated, in the step S1902, a portion of thedisplay screen area is assigned to the occupant who has been granted noinitiative.

In the step S1902, in consideration of at least any one of theauthorization setting information of the occupant who has been grantedthe initiative and the boarding status information of the occupant whohas been granted the initiative, control is performed such that externalinterest information is displayed beyond the area assigned to theoccupant who has been granted no initiative.

In the step S1901, when it is confirmed that the occupant who has beengranted the initiative leaves the vehicle, and the occupant who has beengranted no initiative is seated, in the step S1902, control is performedsuch that the previously used functions are displayed on the screenusing the usage history information of the occupant who has been grantedno initiative.

FIG. 20 is a view showing a button display control in an occupantgaze-based vehicle display and the initiative control system accordingto another embodiment of the present disclosure.

The occupant gaze-based button display control and initiative controlsystem in a vehicle display according to the embodiment of the presentdisclosure includes: a first occupant gaze recognition unit 2001configured to recognize a gaze of a first occupant in a vehicle displayscreen area; and a button display control unit 2003 that performscontrol to display a button on a video content according to the gaze ofthe first occupant.

When the first occupant gaze recognition unit 2001 confirms that thegaze of the first occupant stays on a predetermined screen area for acertain period of time or longer, the button display control unit 2003generates and displays a cursor, and controls the movement and sizechange of the cursor according to the gaze movement of the firstoccupant and the gaze distance of the first occupant.

When the first occupant gaze recognition unit 2001 confirms that thegaze of the first occupant stays on the predetermined screen area for acertain period of time or longer, the button display control unit 2003performs control to display at least any one of a player button and aprogress button of the video content.

The occupant gaze-based button display control in the vehicle displayand initiative control system according to the embodiment of the presentdisclosure further includes a second occupant gaze recognition unit 2002configured to recognize a gaze of a second occupant participating in theuse of the vehicle display, and the button display control unit 2003performs control to display a button for the initiative switch.

The occupant gaze-based button display control and initiative controlsystem in the vehicle display according to the embodiment of the presentdisclosure further includes a display control unit 2004 configured toassign the vehicle display screen area to the first occupant and thesecond occupant when the initiative switch is approved.

The display control unit 2004, with the approval of the first occupant,also displays the external interest information of the second occupanteven in a first area assigned to the first occupant beyond a second areaassigned to the second occupant.

When the second occupant stares at the video content displayed in thefirst area for a preset time or longer, the display control unit 2004performs control to display the video content throughout the vehicledisplay screen area.

FIGS. 21 to 23 are views showing button display control according toanother embodiment of the present disclosure.

Referring to FIG. 21 , when it is confirmed that the occupant stares ata certain area on the screen of the video content played on the vehicledisplay, a desired (for example, necessary, corresponding, appropriate,etc.) button is created.

The button display control unit controls the movement and size of thecursor based on the gaze movement of the occupant and the gaze distanceof the occupant.

The occupant selects a function item by staring at a certain area for apreset time or longer, or by inputting a gesture on a mobile phonescreen connected to a vehicle AVN.

The button display control unit displays the player button (play, pause,move forward, move backward, volume, screen brightness, etc.) and theprogress button, and executes a button display item for a functionexecution by item selection.

Referring to FIG. 22 , when the occupant stares at a left area on thescreen where the video content is played, a cursor is created, and thebutton display control unit changes the cursor to match the appearanceof the widget, confirms that the occupant stares at the left area forthe preset time or longer, and displays the player button and theprogress button.

The button display control unit provides a widget to float the button inan area where the gaze does not stay when the gaze of the occupantmoves.

Referring to FIG. 23 , the button display control unit determines thesize of the cursor differently, depending on the distance between theoccupant and the vehicle display.

FIG. 24 is a view showing initiative switch in another occupant'sparticipation according to another embodiment of the present disclosure.

In a situation where an occupant A is watching the video content playedon the vehicle display screen, when an occupant B participates in theuse of the vehicle display by staring at a particular area, a button forinitiative switch is displayed.

At this time, as the occupant B selects a button for initiative switch,the initiative is provided to the occupant B, or a button to watch thevideo together is displayed.

When the initiative switch is requested by the gaze of the occupant B, abutton for the occupant A, who currently has the initiative, to agree tothe initiative switch is provided on the screen.

When the initiative switch is approved at the request of the occupant B,only the screen of the occupant B may be configured, which will bedescribed with reference to FIG. 25 .

FIG. 25 is a view showing display control based on screen area divisionand gaze information upon another occupant's participation according toanother embodiment of the present disclosure.

The occupant A is watching the video content in a first area A-1 of thevehicle display, and according to the participation of the occupant B, asecond area B-1 of the vehicle display area is assigned to the occupantB.

It is assumed that in the first area A-1, the video content selected bythe occupant A is played, and in the second area B-1, the externalinterest information is displayed.

At this time, according to the approval of the occupant A, the externalinterest information is displayed on the first area A-1 beyond thesecond area B-1.

At this time, when the occupant B stares at the video content for acertain period of time or longer, it is confirmed that the occupant Bhas the intent to watch the video content instead of the externalinterest information, and control is performed such that the videocontent is displayed in the entire area of the vehicle display.

FIG. 26 is a view showing a method of button display control in anoccupant gaze-based vehicle display and initiative control according toanother embodiment of the present disclosure.

The method of button display control in an occupant gaze-based vehicledisplay and initiative control method according to the embodiment of thepresent disclosure includes: a step S2601 for recognizing a gaze of afirst occupant in a vehicle display screen area; and a step S2602 forperforming control to display a button on a video content according tothe gaze of the first occupant.

When it is confirmed in the step S2601 that the gaze of the firstoccupant stays on a predetermined screen area for a certain period oftime or longer, in the step S2602, a cursor is created and displayed,and the movement and size change of the cursor are controlled accordingto the gaze movement of the first occupant and the gaze distance of thefirst occupant.

When it is confirmed in the step S2601 that the gaze of the firstoccupant stays on the predetermined screen area for a certain period oftime or longer, in the step S2602, control is performed such that atleast any one of a player button and a progress button of the videocontent is displayed.

In the step S2601, a gaze of a second occupant participating in the useof the vehicle display is additionally recognized, and in the stepS2602, control is performed such that a button for initiative switch isdisplayed.

The occupant gaze-based button display control in the occupantgaze-based vehicle display and initiative control method according tothe embodiment of the present disclosure further includes a step S2603of confirming the approval of the first occupant according to aninitiative switching request of the second occupant, and performinginitiative switching control.

The occupant gaze-based button display control based on the occupantgaze-based vehicle display and the initiative control method accordingto the embodiment of the present disclosure further includes a stepS2604 of splitting and assigning the vehicle display screen area to thefirst occupant and the second occupant when the initiative switchingrequest is approved, and performing display control using the gazeinformation of the first occupant and the second occupant.

In the step S2604, with the approval of the first occupant, the externalinterest information of the second occupant is also displayed in thefirst area assigned to the first occupant beyond the second areaassigned to the second occupant.

In the step S2604, when the second occupant stares at the video contentdisplayed in the first area for a preset time or longer, control isperformed such that the video content is displayed throughout thevehicle display screen area.

FIG. 27 is a view showing a vehicle display control system fortransmitting external interest information according to embodiment ofthe present disclosure.

The vehicle display control system for transmitting external interestinformation according to the embodiment of the present disclosureincludes: an external interest information confirmation unit 2701configured to confirm external interest information of a first occupantin a vehicle; and a display control unit 2703 configured to performcontrol to display the external interest information of the firstoccupant through a vehicle display.

The external interest information confirmation unit 2701 uses gazeinformation of the first occupant to confirm an external interestobject.

The external interest information confirmation unit 2701 uses voiceinformation and the gaze information of the first occupant to confirmthe external interest object.

The external interest information confirmation unit 2701 confirms theexternal interest object using a gesture information and the gazeinformation of the first occupant.

The display control unit 2703 performs control to display the externalinterest information through at least any one of highlight, areadisplay, contour display, and zoom display.

The vehicle display control system for transmitting external interestinformation according to the embodiment of the present disclosurefurther includes a boarding position recognition unit 2702 configured toconfirm position information of the first occupant and a second occupantin the vehicle, and the display control unit 2703 performs control todisplay the external interest information through a display arearecognizable by the second occupant in consideration of the positioninformation.

FIG. 28A and FIG. 28B are views showing external interest informationdisplay and external interest information display area control accordingto another embodiment of the present disclosure.

Referring to FIG. 28A, a driver A recognizes an interest object (e.g.,bear) viewed through the front windshield of the driving vehicle.

The external interest information confirmation unit confirms theexternal interest information when the gaze of the driver A stays on acertain area for a certain period of time or longer.

When the driver A says “Did you see that, guys?”, the external interestinformation confirmation unit checks the external interest informationbased on the utterance information and gaze information of the driver A.

The external interest information confirmation unit confirms theexternal interest information when the gaze of the driver A stays on acertain area for a certain period of time or longer and performs apredetermined gesture (e.g., finger pointing operation) for indicatingthe external interest information.

The display control unit performs control to display the externalinterest information of the driver A through at least any one ofhighlight, area display, contour display, magnification display.

Referring to FIG. 28A, the display control unit performs control todisplay the area of the external interest object of the driver A throughthe area display on the front windshield, which is a transparentdisplay.

Referring to FIG. 28B, as described above, when the external interestinformation of the driver A is displayed on the front windshield, it isassumed that it is difficult for a passenger B and passenger C, whichare blocked by the driver A and located at the rear seat of the driverA, to easily confirm the corresponding external interest information.

At this time, in consideration of the seating position information ofthe passengers obtained through the boarding position recognition unit,the display control unit performs control to display the externalinterest information on the transparent display of the left side windowlocated near the sitting position of the passenger B and to display theexternal interest information on the transparent display of the rightside window located near the sitting position of the passenger C, sothat the passenger B and the passenger C can easily confirm the externalinterest information.

The boarding position recognition unit recognizes the position of theoccupant in the vehicle using at least any one of a seat sensor and anindoor camera, and the display control unit uses the position todetermine a display area (the aforementioned rear seat left side windowdisplay area and rear seat right side window display area) that thepassengers can comfortably see, and displays the external interestinformation in the determined area.

FIG. 29 is a view showing a vehicle display control method fortransmitting external interest information according to anotherembodiment of the present disclosure.

The vehicle display control method for transmitting external interestinformation according to the embodiment of the present disclosureincludes: a step S2901 for confirming external interest information of afirst occupant in a vehicle; a step S2902 for confirming the boardingpositions of the first occupant and the second occupant in the vehicle;and a step S2903 for performing control to display the external interestinformation through a vehicle display in consideration of the boardingpositions.

In the step S2901, the external interest information is confirmed usingat least any one of gaze information, voice information, and gestureinformation of the first occupant.

In the step S2903, control is performed such that the external interestinformation is displayed through at least any one of highlight, areadisplay, contour display, or zoom display.

In the step S2903, when it is confirmed by using the boarding positionthat the field of view of the second occupant is not blocked, theexternal interest information confirmed by the first occupant beyond afirst display area is displayed in the first display area.

In the step S2903, when it is confirmed by using the boarding positionthat the field of view of the second occupant is blocked, the externalinterest information confirmed by the first occupant beyond the firstdisplay area is displayed in a second display area that is differentfrom the first display area.

FIG. 30 is a view showing a vehicle display control system forsynthesizing and displaying external interest information according toanother embodiment of the present disclosure.

The vehicle display control system for synthesizing and displayingexternal interest information according to the embodiment of the presentdisclosure includes: an external interest information confirmation unit3001 configured to confirm an interest object outside a vehicle in whichan occupant in the vehicle is interested; an occupant operationinformation recognition unit 3002 configured to recognize operationinformation of the occupant in the vehicle associated with the rotationof external interest information; and a display control unit 3003configured to synthesize and display a 3D image for the interest objectoutside the vehicle, and to rotate and display the 3D image according tothe operation information.

The external interest information confirmation unit 3001 confirms aninterest object located in an area where the gaze of the occupant in thevehicle stays for a preset time or longer.

The occupant operation information recognition unit 3002 recognizes atleast any one of a touch input, gesture input, and voice input as theoperation information.

The display control unit 3003 determines the degree of rotation of the3D image according to the operation information, rotates the 3D image todisplay the rotated 3D image, and displays the 3D image by restoring the3D image to the actual position of the interest object when there is nofurther operation for a preset time or longer.

FIGS. 31A, 31B and 32 are views showing external interest informationrotation and display according to another embodiment of the presentdisclosure.

Referring to FIG. 31A and FIG. 31B, the gaze information of the occupantin the vehicle is used to determine the external interest object amongthe objects viewed through the vehicle windshield, and the correspondinginterest object is synthesized into a 3D form and displayed.

At this time, the external interest information synthesized in 3D formis displayed in the front windshield area, which is a transparentdisplay.

When the occupant in the vehicle drags the external interest informationsynthesized in 3D form to the right using touch or gesture, the externalinterest information synthesized in 3D form is rotated to the right bythe value of the drag.

When a preset time passes after the external interest informationsynthesized in 3D form is rotated and displayed according to theoperation of the occupant in the vehicle, the position of the externalinterest information synthesized in the 3D form is restored to theposition of the actual external interest object.

Referring to FIG. 32 , the external interest object is synthesized anddisplayed in 3D form, and control is performed such that the 3D objectis rotated by 360 degrees up and down/left and right through multimodalinteractions such as touch, gesture, and voice recognition of theoccupant in the vehicle.

FIG. 33 is a view showing a vehicle display control method forsynthesizing and displaying external interest information according toanother embodiment of the present disclosure.

The vehicle display control method for synthesizing and displayingexternal interest information according to the embodiment of the presentdisclosure includes: a step S3301 for confirming external interestinformation of an occupant in a vehicle; a step S3302 for recognizingoperation information of the occupant associated with the rotation ofthe external interest information; and a step S3303 for performingcontrol to synthesize and display the external interest information as3D image, and to rotate and display the 3D image according to theoperation information.

In the step S3301, the external interest information for an externalobject located in an area where the gaze of the occupant in the vehiclestays for a preset time or longer is confirmed.

In the step S3302, at least any one operation information of touchinput, gesture input, and voice input is recognized.

In the step S3303, the 3D image is rotated and displayed according tothe degree of rotation determined based on the operation information,and when there is no further operation for a preset time or longer, the3D image is displayed by restoring the 3D image to the actual positionof the interest object.

FIG. 34 is a view showing a vehicle display control system for storingand sharing external interest information according to anotherembodiment of the present disclosure.

The vehicle display control system for storing and sharing externalinterest information according to the embodiment of the presentdisclosure includes: an occupant operation recognition unit 3401configured to receive operation information for an video storage requestfor an external interest object of a vehicle occupant; and a displaycontrol unit 3402 configured to perform video storage according to theoperation information for the external interest object at which thevehicle occupant stares through a vehicle windshield configured as atransparent display.

The occupant operation recognition unit 3401 receives the operationinformation that is input using at least any one of gaze information,voice information, gesture information, and touch information.

The display control unit 3402 performs control to display informationabout the external interest object as AR information.

The display control unit 3402 captures and stores an external foregroundand the AR information at the same time.

The display control unit 3402 controls the display for screen capture bywhite-out during a preset time.

The vehicle display control system for storing and sharing externalinterest information according to the embodiment of the presentdisclosure further includes: an external interest informationtransmission unit 3403 configured to transmit a video stored accordingto the operation information to at least any one of a divided area in avehicle display, a mobile terminal of the vehicle occupant, and a mobileterminal of another user.

The display control unit 3402 performs control to divide and assign adisplay area to a plurality of occupants in the vehicle, store theexternal interest information video according to the video storagerequest made within each area, and display the stored video in apredetermined area of an assigned display area.

The display control unit 3402 transmits control information to move thetransparent display in consideration of access control priorities of theplurality of occupants.

The display control unit 3402 performs control to move the videoinformation stored in a predetermined area of a first occupant to apredetermined area of a second occupant according to the occupantrequest and sharing

setting information.

Hereinafter, in FIGS. 35A, 35B, 35C, 36A, 36B, 37A, 37B, 37C, 38A, 38B,38C, it is assumed that a rear seat left side occupant in a vehiclestares at the outside through a rear seat left side window transparentdisplay.

FIG. 35A and FIG. 35C are views showing a process of capturing theexternal interest information according to another embodiment of thepresent disclosure.

Referring to FIG. 35A, the display control unit 3402 displays fixedinformation including at least any one of position information, weather,battery information, and transparency of a transparent display throughthe transparent display.

Referring to FIG. 35B, the occupant operation recognition unit 3401confirms the area where the gaze of the occupant stays for a certainperiod of time or longer through the rear seat left side window, and thedisplay control unit 3402 displays AR information based on the gazetracking information.

At this time, the display control unit 3402 displays the information ofa mountain name and height, weather, distance, and travel time (the timerequired for the vehicle to travel from the current position to theentrance to the mountain, the time required to climb the mountain fromthe entrance to the mountain to the top of the mountain, etc.) as ARinformation.

Referring to FIG. 35C, the occupant operation recognition unit 3401recognizes various pieces of operation information of the occupant todetermine the intent of storing the external interest information.

That is, the occupants request the storage of the corresponding externalinterest information in a variety of ways, including gaze recognitionand a touch of a save button within a mobile device app, gazerecognition and a bottom swipe touch within the mobile device app, gazerecognition and a specific voice command (e.g., save it or capturethis), gaze recognition and a specific gesture, and gaze recognition anda display swipe down touch.

Referring to FIG. 35B, the display control unit 3402 simultaneouslycaptures the external foreground and AR information (as another example,it is also possible to cut and store only the area of the correspondingexternal interest information) according to the storage request receivedfrom the occupant operation recognition unit, and in this case, thecapture information may include a photograph, video, AR information, GPSposition information.

The display control unit 3402 whites the screen out for a preset time(e.g., 1 second) when capturing the screen.

The display control unit 3402 performs control to display the screenshotscreen in a specific area of the display (e.g., bottom left) with acertain size (e.g., 15 cm×15 cm) for a certain period of time (e.g., 3seconds).

FIG. 36A and FIG. 36B are views showing a process of sharing theexternal interest information with a smartphone according to anotherembodiment of the present disclosure.

According to the gaze tracking of the occupant, in a situation where theexternal interest information is displayed in AR through the rear seatside window transparent display, when the occupant operation recognitionunit 3401 receives information that the occupant touches a particularbutton using his/her mobile terminal connected to the vehicle, thedisplay control unit 3402 captures a screen according to a preset method(e.g., captures the external foreground and AR information at the sametime), and when captured, the screen is whited out for the preset time(e.g., 1 second).

The external interest information transmission unit 3403 transmits thecaptured screen to the mobile termina of the occupant, and the occupantcan check whether a desired capture image has been received normally,through the app of the mobile terminal (e.g., album).

Through in-album manipulation, the occupant can flip through the imagedisplayed in the mobile terminal, and presses a specific button (sharebutton) to transmit the image to be shared to someone else's smartphoneor a vehicle.

FIG. 37A to FIG. 37C are views showing a process of storing andconfirming external interest information for each occupant throughtransparent display sliding according to another embodiment of thepresent disclosure.

Referring to FIG. 37A, an occupant A is seated on the rear left seat ofthe driving vehicle, an occupant B is seated on the rear right seat ofthe driving vehicle, and the occupant A and the occupant B stare at theoutside through the rear seat side window.

The rear seat side window is configured as a transparent display andprovides an area assigned for each occupant.

For example, in consideration of the viewing angle of the occupant, afirst area A-1 is assigned to the occupant A and a second area B-1 isassigned to the occupant B.

As another example, it is possible to assign a display area based on theposition of the external object at which the occupants currently stare.In other words, as an example different from that shown in FIG. 37A,when the occupant A stares at the isolated spire as the interest object,and the occupant B stares at the double spire as the interest object, itis also possible to assign the second area to the occupant A and thefirst area to the occupant B.

Referring to the illustrations in FIG. 37A, the occupant A stares at thedouble spire shown beyond the first area A-1 as the interest object, andwhen the occupant A performs swipe down on the display, the displaycontrol unit 3402 stores and displays the captured image in a specificpersonalization area at the bottom of the display area. In addition,since the occupant B is a person who is further away than the occupant Afrom the rear seat left side window, the occupant B transmits thecapture intent through the gaze recognition and air gesture, and thedisplay control unit 3402 stores and displays the capture image asrequested by the occupant B in a specific personalization area at thebottom of the display area.

Referring FIG. 37B, it is assumed that in an autonomous drivingsituation, the driving control depends on the autonomous driving mode,an occupant C seated on the driver's seat is viewing the outside scenerythrough the left side window, and the occupant C has the left-sidewindow display access control with a priority higher than that that ofthe occupant B (It is assumed that the priority for display accesscontrol is higher, the closer the display from the boarding position).

At this time, the occupant C slides forward the transparent displaycorresponding to the second area B-1, and thus, the second area B-1 isdisposed at the left side of the occupant C, and the second area B-1 isassigned to occupant C as a third area C-1.

The occupant C requests the storage of the external interest informationthrough a display swipe down, and the display control unit 3402 displaysthat the external interest information is stored as a capture image in aspecific area at the bottom of the display.

Referring to FIG. 37C, in consideration of a face ID camera placed inthe vehicle and the seat position information, authentication isperformed for each occupant, and when authentication is completed, apersonalized content history is displayed in an area at the bottom ofthe display area assigned to each occupant.

The content history display displays the image/video files in the orderin which they are captured.

FIG. 38A to FIG. 38C are views showing the sharing of the externalinterest information between displays according to another embodiment ofthe present disclosure.

Referring to FIG. 38 , a first area A-1 is assigned to an occupant A anda second area B-1 is assigned to an occupant B, and the content historyof each occupant is displayed in a specific area at the bottom of eacharea.

Referring to FIG. 38A, the occupant B selects a certain content in thecontent history (long press, which can be selected through gesture), anddrags the object and requests the transfer of the object to the displayof the occupant A.

Referring to FIG. 38B, in consideration of the information sharingauthorization setting information of the occupant A, when theinformation sharing is approved, the display control unit 3402 moves thecorresponding particular content to the personalized area of theoccupant A and displays it.

At this time, the display control unit 3402 performs control to allownot only the same side (rear seat left side window) display but thefront or opposite side (rear seat rear side window) display based on theoccupant's request and information sharing setting to share the externalinterest information according to the directionality of the drag anddrop.

Referring to FIG. 38C, a particular content shared by the operation ofthe occupant B is added and stored into the personalization area of theoccupant A.

FIG. 39 is a view showing a vehicle display control method for storingand sharing external interest information according to anotherembodiment of the present disclosure.

The vehicle display control method for storing and sharing externalinterest information according to the embodiment of the presentdisclosure includes: a step S3901 for receiving a video storage requestfor an external interest object of a vehicle occupant; and a step S3902for performing a video storage for the external interest object at whichthe vehicle occupant stares through a vehicle windshield configured as atransparent display and performing display control.

In the step S3901, a video storage request input using at least any oneof gaze information, voice information, gesture information, and touchinformation is received.

In the step S3902, display control is performed such that informationabout the external interest object is displayed as AR information.

In the step S3902, an external foreground and AR information arecaptured and stored at the same time.

In the step S3902, display control for screen capture is performed bywhite-out a screen during a preset time.

The vehicle display control method for storing and sharing externalinterest information according to the embodiment of the presentdisclosure further includes a step S3903 for sharing the externalinterest information by transmitting the stored video to at least anyone of a divided area in a vehicle display, a mobile terminal of thevehicle occupant, and a mobile terminal of another user.

In the step S3902, control is performed such that the display area isdivided and assigned to a plurality of occupants in the vehicle, and anexternal interest information video is stored according to the videostorage request made within each area, and the stored video is displayedin a predetermined area of the assigned display area.

In the step S3902, in consideration of access control priorities of aplurality of occupants. display control is performed such that thetransparent display is moved.

In the step S3902, control is performed such that video informationstored in a predetermined area of a first occupant is moved to apredetermined area of a second occupant according to the occupantrequest and sharing

of setting information.

FIG. 40 is a view showing a vehicle display area control system forexternal interest information according to another embodiment of thepresent disclosure.

The occupant gaze-based external interest information display areadetermination and vehicle display control system according to theembodiment of the present disclosure includes: an external interestinformation display area confirmation unit 4001 configured to confirm adisplay area for displaying external interest information of a pluralityof occupants in the vehicle; and a display control unit 4002 configuredto adjust the display area of the external interest information, inconsideration of at least any one of importance of the external interestinformation and the initiative granted to the plurality of occupants.

The display control unit 4002 moves the display area of the externalinterest information of a second occupant so that the display area ofthe external interest information of the second occupant does notoverlap with the display area of the external interest information of afirst occupant, when it is determined that the importance of theexternal interest information of the first occupant is higher than theimportance of the external interest information of the second occupant.

The display control unit 4002 moves the display area of the externalinterest information of the second occupant when the first occupant hasthe initiative and it is determined that the display area of theexternal interest information of the first occupant and the display areaof the external interest information of the second occupant overlap witheach other in a certain part or more.

The display control unit 4002 performs control to display outlineinformation about the original position of the external interestinformation of the second occupant.

In consideration of the gaze information of the first occupant, thedisplay control unit 4002 restores the display area of the externalinterest information of the second occupant to the area before moving.

FIG. 41A to FIG. 41C are views showing a process of controlling avehicle display area for external interest information according toanother embodiment of the present disclosure.

Referring to FIG. 41A, a driver A and a passenger B look at externalobjects through a front windshield of a vehicle.

At this time, when the driver A stares at a forward vehicle located inthe area of □ for a preset time or longer, driving-related information(e.g., distance from the forward vehicle and AR route guidanceinformation based on navigation information) is displayed on the frontwindshield, which is a transparent display, according to the result oftracking the gaze of the driver A.

The passenger B stares at a store (café) located in the area of □ for apreset time or longer, and accordingly, the information about the store(e.g., phone number, the number of floors, menu) is displayed on thefront windshield, which is a transparent display.

Referring to FIG. 41B, as the driver A looks at a stopped vehiclelocated in the area of □, the display area for the external interestinformation of the passenger B overlaps with the display area for theexternal interest information of the driver A.

The display control unit 4002 changes the position of the display areafor the external interest information of the passenger B in order toprevent the field of view of the initiative owner, the driver A, frombeing blocked by this overlapping phenomenon.

That is, while the passenger B is staring at the information about thecafé located on the first floor of the building, the driver A isprovided with rear-facing warning information, and when the driver Astares at another vehicle, according to the priority of the initiativeand information, the display control unit 4002 performs control to moveand display the café information to the upper area of the building.

At this time, the display control unit 4002 continuously displays theposition of the previous interest information in the form of a dottedline, so that the position can be recognized.

Referring to FIG. 41C, when the rear side warning information providedto the driver A is no longer transmitted, or when the driver A moveshis/her gaze to another object (forward vehicle), the display controlunit 4002 restores the position of the display area for the externalinterest information of the passenger B.

FIG. 42 is a view showing a method of controlling a vehicle display areafor external interest information according to another embodiment of thepresent disclosure.

The occupant gaze-based external interest information display areadetermination and a vehicle display control method according to theembodiment of the present disclosure includes: a step S4201 forconfirming an area for displaying external interest information of anoccupant in a vehicle; and a step S4202 for adjusting the position ofthe area for displaying the external interest information, inconsideration of at least any one of importance of the external interestinformation and information about the initiative granted to theoccupant.

In the step S4202, when it is determined that the importance of theexternal interest information of a first occupant is higher than theimportance of the external interest information of a second occupant,the positions of the display area of the external interest informationof the second occupant is moved, so that the display area of theexternal interest information of the second occupant does not overlapwith the display area of the external interest information of the firstoccupant.

In the step S4202, when the first occupant has the initiative, and it isconfirmed that the display area for the external interest information ofthe first occupant and the display area for the external interestinformation of the second occupant overlap with each other in a certainpart or more, the display area for the external interest information ofthe second occupant is moved.

In the step S4202, control is performed such that outline informationabout the original position of the external interest information of thesecond occupant is displayed.

In the step S4202, in consideration of the gaze information of the firstoccupant, the display area for the external interest information of thesecond occupant is restored to the area before moving.

FIG. 43 is a view showing a vehicle display control system fordisplaying and storing external interest information according toanother embodiment of the present disclosure.

The vehicle display control system for displaying and storing externalinterest information according to the embodiment of the presentdisclosure includes: an external interest object confirmation unit 4301configured to confirm an external interest object in which an occupantin a driving vehicle is interested; and a display control unit 4302configured to perform control to display external interest informationabout the external interest object in a first display area, to move anddisplay the external interest information to a second display areaaccording to a driving situation, and to store the external interestinformation in an external interest information storing unit 4303.

The external interest object confirmation unit 4301 confirms theexternal interest object using at least any one of gaze information,voice information, and gesture information of the occupant.

When the gaze of the occupant departs from a display area for theexternal interest information, the display control unit 4302 performscontrol to simplify the external interest information, move thesimplified external interest information to the second display area,which is a side window display, and to display the simplified externalinterest information.

The display control unit 4302 performs control such that the simplifiedexternal interest information is accumulated in chronological order anddisplayed in the second display area.

The vehicle display control system for displaying and storing externalinterest information according to the embodiment of the presentdisclosure further includes a destination setting request receiving unit4304 configured to receive a destination setting request for thesimplified external interest information.

The vehicle display control system for displaying and storing externalinterest information according to the embodiment of the presentdisclosure further includes a route setting unit 4305 configured toestablish a route to an external point of interest selected inaccordance with the destination setting request.

FIG. 44A and FIG. 44B are views showing external interest informationdisplay according to another embodiment of the present disclosure.

Referring to FIG. 44A, a driver in a driving vehicle stares at aninterest object (parking lot) for a preset time or longer among externalobjects viewed through a front windshield configured as a transparentdisplay, and accordingly, external interest information (e.g.,information about the name, position and cost of the parking lot) isdisplayed as AR information.

At this time, the display control unit 4302 performs control to confirmthe external interest object using voice information, head movementinformation, and gesture information in addition to occupant gazeinformation, and to display information about the external interestobject.

Referring to FIG. 44A, the display control unit 4302 simplifies thecorresponding external interest information (e.g., simplifiedinformation such as the distance from the current position, parkingcosts, etc.) when the driver's field of view departs from a display areafor the external interest information, or when the external interestobject becomes out of the field of view of the driver as the drivingvehicle moves, and displays the simplified external interest informationin the form of a log.

At this time, the display control unit 4302 performs control to displaythe simplified external interest information in a predetermined area ofa front display or a side window display.

Referring to FIG. 44B, when the external point of interest movesrelatively and is out of the driver's field of view as the vehicletravels, the display control unit 4302 displays the external interestinformation on the side window display by time. That is, the displaycontrol unit 4302 accumulates and displays the external interestinformation in the order of time elapsed.

At this time, the display control unit 4302 performs control to displayinformation about the external interest object, located on the frontleft side of the driving vehicle, through the left side window display,and to display the information about the external interest object,located on the front right side of the driving vehicle, through theright side window display.

When receiving a destination setting request for particular externalinterest information among the plurality of stored external interestinformation, the route setting unit 4305 sets a path from the currentposition to the position of the corresponding external interestinformation.

The destination setting request receiving unit 4304 receives adestination setting request using touch, voice, air gesture, and amobile device for external interest information stored in the sidewindow display.

FIG. 45 is a view showing a vehicle display control method fordisplaying and storing external interest information according toanother embodiment of the present disclosure.

The vehicle display control method for displaying and storing externalinterest information according to the embodiment of the presentdisclosure includes: a step S4501 for confirming an external interestobject of a driving vehicle occupant; and a step S4502 for performcontrol to display external interest information for the externalinterest object in a first display area, and moving the externalinterest information to a second display area according to a drivingsituation to display the moved external interest information.

In the step S4501, the external interest object is confirmed using atleast any one of the gaze information, voice information, and gestureinformation of the occupant.

In the step S4502, when the gaze of the driving vehicle occupant departsfrom the display area for the external interest information, control isperformed such that the external interest information is simplified, andthe simplified external interest information is moved to the seconddisplay area which is the side window display and displayed.

In the step S4502, control is performed such that the simplifiedexternal interest information is accumulated in chronological order andis displayed in the second display area.

The vehicle display control method for displaying and storing externalinterest information according to the embodiment of the presentdisclosure further includes a step S4503 for receiving a destinationsetting request for the simplified external interest information andsetting a route to an external point of interest selected according tothe destination setting request.

FIG. 46 is a block diagram illustrating a system for controlling avehicle display according to still another embodiment of the presentdisclosure.

The system for controlling a vehicle display according to the presentembodiment of the present disclosure incudes a traveling environmentrecognition unit 4610 that acquires traveling environmental informationof a traveling vehicle, a control unit 4620 that generates controlinformation associated with a displaying position of additionalinformation that is to be presented through a display according to thetraveling environmental information, and a display unit 4630 thatdisplays the additional information on a display region according to thecontrol information.

The traveling environment recognition unit 4610 collects the travelingenvironmental information associated with a state of a road surface infront of the traveling vehicle.

The control unit 4620 predicts a vehicle traveling behavior according tothe traveling environmental information. According to a result of theprediction, the control unit 4620 computes an adjustment value of thedisplaying position of the additional information that is beingcurrently presented through the display.

At this point, the traveling environment recognition unit 4610 transmitstraveling information (for example, a traveling speed) of the travelingvehicle, and the control unit 4620 computes the adjustment value,considering the traveling information and the traveling environmentalinformation. In a case where vehicles that travel at a speed of 30 km/hand at a speed of 50 km/h, respectively, go over the same speed bump,the external objects of interest that appear across side-window displaysof the vehicles are also different from each other in a moving speed inan upward-downward direction. Therefore, the control unit 4620 computesthe adjustment value, considering the traveling information, as well asthe traveling environmental information. At this point, it is possibleto compute the adjustment value, considering the traveling historyinformation of each of the vehicles, for example, predicting how muchthe vehicle needs to reduce a speed per hour to go over the speed bump.

The traveling environment recognition unit 4610 transmits information onan occupant inside of the vehicle, and the control unit 4620 computesthe adjustment value, considering the information on the occupant. Forexample, the traveling environment recognition unit 4610 transmits aposition of a vehicle occupant, information on a region of the displayregion that is allocated to each of the vehicle occupant, a sittingheight of the vehicle occupant, a position at which the vehicle occupantgazes toward the external object of interest, a direction in which thevehicle occupant gazes toward the external object of interest, and thelike. The control unit 4620 computes an adjustment value for controllingthe displaying position of the additional information on the externalobject of interest, considering these pieces of information on theoccupant.

FIG. 47 is a view illustrating a vehicle display control processaccording to the present embodiment of the present disclosure.

With reference to FIG. 47 , information on the state of the road surfacein front of the traveling vehicle through the traveling environmentrecognition unit 4610 including a camera 4730. At this point, it ispossible to collect the traveling environmental information by utilizinga sensor in addition to the camera, by using navigation information, orby collecting external-environment information through V2Icommunication.

In FIG. 47 , it is assumed that it is recognized through a vehicledisplay (a side-window display is illustrated as an example thereof)that a region 4710 and a region 4720 are allocated to Occupant C andOccupant B, respectively. The control unit 4620 performs control in sucha manner that the additional information (AR information) on a firstexternal object of interest (double spire) at which Occupant C gazes andthe additional information (AR information) on a second external objectof interest (isolated spire) at which Occupant B gazes are presented onthe vehicle display. The display unit 4630 displays the additionalinformation (with reference to FIG. 47 , the display unit 4630 displaysa contour of each of the corresponding external objects of interest,using lines in sky blue and displays a name of each of the correspondingexternal objects of interest) on each of the first and second externalobjects of interest.

In a situation where, as can be seen from FIG. 47 , the vehicle alsogoes up the speed bump, the display physically moves upward, togetherwith the vehicle. At this point, in accordance with a position of theexternal object of interest, the control unit 4620 performs control insuch a manner that the displaying position of the additional information(AR information) is adjusted to a relatively lower height as illustratedin than the displaying position. According to this control, the displayunit 4630 displays the additional information at the displayingposition.

In a situation where, as can be seen from FIG. 47 , the vehicle goesdown the speed bump, the display also physically moves downward,together with the vehicle. At this point, in accordance with theposition of the external object of interest, the control unit 4620performs control in such a manner that the displaying position of theadditional information (AR information) is adjusted to a relativelyhigher height than the displaying position.

According to this control, the display unit 4630 displays the additionalinformation at the displaying position.

According to the present embodiment of the present disclosure, thevehicle traveling behavior is predicted (for example, the vehicletraveling behavior is predicted to determine a height of the speed bumpover which the vehicle will go) by analyzing information acquired usinga camera, a sensor, navigation map information, and V2I communication.Accordingly, it is possible to generate, in advance, information forcontrolling the displaying position of the additional information thatis to be presented on the display. Accordingly, in a traveling situationwhere the vehicle goes over the speed bump, it is possible to present inreal time the additional information on the external object of interestthat is located across a side window, on the side-window display in amanner that corresponds to the position of the external object ofinterest.

According to the present embodiment of the present disclosure, thepieces of information on the occupant, such as a sitting position of thevehicle occupant, the sitting height of the vehicle occupant, theposition at which the vehicle occupant gazes toward the external objectof interest, and the direction in which the vehicle occupant gazestoward the external object of interest, are acquired, and accordingly, ashaking compensation information for information on the external objectof interest is generated. For example, in FIG. 47 , it is assumed thatOccupant C is an occupant, having a sitting height of 1 m, who sits on aseat positioned a 30 cm away from the allocation region 4710 of theside-window display and that Occupant B is an occupant, having a sittingheight of 1 m, who sits on a seat positioned 3 m away from theallocation region 4720 of the side-window display. The control unit 4620acquires the position of the vehicle occupant and the position at whichthe vehicle occupant gazes toward the external object of interest, andaccordingly generates information on displaying-position adjustment ofthe additional information in a manner that varies from one region toanother. That is, in a situation where the vehicles go over the samespeed bump having a height of 50 cm, control is also performed in such amanner that the additional information on the external object ofinterest at which each occupant gazes is also in detail displayed bydifferentiating the degree to which the displaying position of theadditional information is adjusted to a low height in the region 4710allocated to Occupant C and the degree to which the displaying positionof the additional information is adjusted to a low height in the region4720 allocated to Occupant B.

FIG. 48 is a flowchart illustrating a method of controlling a vehicledisplay according to still another embodiment of the present disclosure.

The method of controlling a vehicle display according to the presentembodiment of the present disclosure includes Step S4810 of collectingtraveling-associated information and predicting the vehicle travelingbehavior, Step S4820 of computing the adjustment value of the displayingposition of the additional information associated with the externalobject of interest inside of and outside of the display, and Step S4830of displaying the additional information.

In Step S4810, information on the state of the road surface in front ofthe traveling vehicle is collected using at least one of the camera, thesensor, and the navigation map, and the vehicle traveling behavior ispredicted by analyzing the collected information.

In Step S4810, traveling speed information of the vehicle andinformation on predicted reduction of the traveling speed immediatelybefore the speed bump are acquired. In Step S4820, the adjustment valueof the displaying position of the additional information that varieswith the vehicle traveling behavior is computed on the basis of theacquired traveling speed information and information on predictedreduction of the traveling speed immediately before the speed bump. Forexample, as described above, in a case where the vehicle travelingbehavior, such as going over the speed bump, is predicted, while thevehicle goes up the speed bump, control is performed in such a mannerthat the additional information is presented at a position, relativelylower than a previous position, by adjusting a position at which theadditional information of the external object of interest that islocated across the side-window display is to be presented on theside-window display. Moreover, while the vehicle goes down the speedbump, control is performed in such a manner that the additionalinformation is presented at a position, relatively higher than aprevious position (at a point in time when the vehicle goes up thebump), by adjusting the position at which the additional information ofthe external object of interest that is located across the side-windowdisplay is to be presented on the side-window display.

FIG. 49 is a block diagram illustrating a system for controlling avehicle display according to still another embodiment of the presentdisclosure.

The system for controlling a vehicle display according to the presentembodiment of the present disclosure includes an object-of-interestrecognition unit 4910 that recognizes the object of interest outside ofthe vehicle in which the vehicle occupant is interested, a control unit4920 that performs control in such a manner that the additionalinformation is presented in a composited manner through the vehicledisplay, using 3D information on the object of interest, and a displayunit 4930 that displays the additional information.

The object-of-interest recognition unit 4910 receives image informationfrom an image acquisition device that captures an image of anenvironment outside of the vehicle and recognizes the object of interestusing monitoring information on a user.

The control unit 4920 generates the 3D information on the object ofinterest using distance information received from the image acquisitiondevice and zoom setting information.

The control unit 4920 receives the 3D information on the object ofinterest from a server of a service provider, and performs control insuch a manner that the 3D information is displayed in a compositemanner.

FIGS. 50A and 50B are views illustrating that size information andvolume information of the object of interest outside of the vehicleaccording to the present embodiment of the present disclosure aredisplayed in a converted manner.

With reference to FIG. 50A, the camera that is arranged on a lateralside of the vehicle captures an image of the environment outside of thevehicle and transmits the captured image to the object-of-interestrecognition unit 4910. The object-of-interest recognition unit 4910 thatincludes at least one of a camera, a gesture acquisition unit, and atouch signal acquisition unit that are arranged inside the vehiclechecks which object in the environment outside of the vehicle theoccupant inside of the vehicle regards as the object of interest.

When the occupant inside of the vehicle is checked as being interestedin a building under construction in an external environment, which iscovered with a tent and is located cross the side-window display, thecontrol unit 4920 retrieves information on the object of interest andperforms control in such a manner that converted information on thebuilding is presented as the additional information in a compositedmanner through a transparent display (the display unit 4930) arranged onthe side window. At this point, 3D data on a height, a width, and adepth of the building may be displayed in a composited manner on aper-region basis, and a construction completion date (Sep. 25, 2022) maybe displayed as the additional information, in a composite manner. Whenacquiring an image, the object-of-interest recognition unit 4910provides the size information and the volume information of the externalobject of interest using a distance to a multi-camera that captures theimage of the environment outside of the vehicle, and using the zoomsetting information. The control unit 4920 performs control in such amanner that 3D AR information on the building that is currently underconstruction is presented through the transparent display. In the samemanner as if the construction of the building were completed, thedisplay unit 4930 displays the 3D AR information on the building in acomposited manner, and presents the size (width, height, and depth)information, the volume information and width information through aside-window transparent display.

With reference to FIG. 50B, the camera arranged on the lateral side ofthe vehicle captures the image of the environment outside of the vehicleand transmits the captured image to the object-of-interest recognitionunit 4910. It is assumed that the object of interest is a building inthe environment outside of the vehicle that is under construction. Thecontrol unit 4920 is provided with final 3D data on the buildingcurrently under construction from a real estate service provider andperforms control in such a manner that the shape of the building ofwhich the construction is completed is displayed as the additionalinformation (AR information). The display unit 4930 presents thebuilding of which the construction is completed, through the side-windowdisplay. At this point, structure information and shape information ofthe building, as well as the size information and the volume informationof the building, are presented. The control unit 4920 adjusts an angleof information that needs to be presented through the side-windowtransparent display, using the direction in which the occupant inside ofthe vehicle gazes at the object of interest, and a location of thevehicle. In addition, the control unit 4920 recognizes the needs of theuser through a touch signal, a gesture signal, and a voice signal of theuser that are received through a separate input unit. Moreover, byrotating the object of interest outside of the vehicle, the control unit4920 performs display control in such a manner that the user can checkwhether or not his/her needs are satisfied. That is, in a situationwhere, as illustrated in FIG. 50B, in practice, the front surface of thebuilding of which the construction is completed and which appears acrossthe side window is displayed in a composited manner, it is assumed that,through a voice signal of the user, the user says, “Show me the rearsurface of the building.” At this point, the control unit 4920 enablesthe side-window display to be used as a screen, by performingelectrochromic discoloration on the side-window transparent display.Moreover, the control unit 4920 generates image information on the rearsurface of the corresponding building using 3D data on the correspondingbuilding and performs control in such a manner that the generated imageinformation is presented through the side-window display.

FIG. 51 is a flowchart illustrating a method of controlling a vehicledisplay according to still another embodiment of the present disclosure.

The method of controlling a vehicle display according to the presentembodiment of the present disclosure includes Step S5110 of recognizingthe object of interest outside of the vehicle, Step S5120 of convertinginformation on the external object of interest, and Step S5130 ofpresenting information resulting from the conversion through the vehicledisplay.

In Step S5110, the environment outside of the vehicle is recognizedusing the multi-camera that captures the image of the environmentoutside of the vehicle, and which external object of interest theoccupant inside of the vehicle is interested in is recognized usingmonitoring information (for example, the direction in which the occupantinside of the vehicle gazes at the object of interest, the touch signal,and the gesture signal) on the occupant inside of the vehicle.

In Step S5120, 3D data on the object of interest is generated orretrieved, and the control information is generated in such a mannerthat the information on the object of interest that is located acrossthe vehicle display when viewed from the vehicle occupant is displayedin a composited manner. At this point, the size information and thevolume information of the object of interest are converted using adistance to the object of interest and zoom information. In addition,the information on the object of interest is retrieved from a database,or is provided from the server of the service provider. Thus, theadditional information, such as size information or a volumeinformation, is generated.

FIG. 52 is a block diagram illustrating a system for controlling avehicle display according to still another embodiment of the presentdisclosure.

The system for controlling a vehicle display according to the presentembodiment of the present disclosure includes a content movement inputunit 5210 that receives a request for movement of content among aplurality of display regions, a control unit 5220 that generates acontent movement control command according to the request for themovement, and a display unit 5230 that displays the content according tothe content movement control command.

The content movement input unit 5210 receives the request for themovement of the content by receiving a user request through at least oneof gesture, voice, inputting through a home button, and inputtingthrough a smartphone.

The control unit 5220 determines a content reproduction region,considering allocation information of a content display region accordingto the request for the movement, and accordingly generates the contentmovement control command.

The control unit 5220 generates a control command for controlling atleast one of sound reproduction and seat rotation, considering movementinformation of the content reproduction region.

FIGS. 53A and 53B are views illustrating a display presentation regionand an example of seat control when moving content according to thepresent embodiment of the present disclosure.

With reference to FIGS. 53A and 53B, the occupant inside of the vehicle(a portion indicated by a green circle) watches the content using afront-side display. At this point, the occupant inside of the vehiclewatches an image on a front-side projection using a projection displaytechnique, and audio sound is reproduced through left and rightspeakers. With reference to FIGS. 53A and 53B, the image is moved byrecognizing a gesture of the occupant inside of the vehicle, byperforming inputting through a button on a screen, or by performinginputting through the smartphone. The image is moved toward a front-sidedisplay of the transparent display in order to minimize a gap betweentwo displays. With reference to FIGS. 53A and 53B, video content that isbeing displayed on the front-side display is moved to the side-windowdisplay in such a manner that it is continuously reproduced. At thispoint, a reproduction region varies according to a video arrangementposition. It is assumed that the side-window display is divided into tworegions, a first region and a second region. Content, when dropping onto the first region or the second, is reproduced on the first region orthe second region, respectively. The content, when dropping in a mannerthat straddles the border between the first region and the secondregion, is reproduced on the entire screen (the first region and thesecond region).

With reference to With reference to FIGS. 53A and 53B, the image ismoved from a projection display to the transparent display, and the seatrotation is performed. At this point, by adjusting a speaker phase, anaudio output source is moved together with the image, and sound isreproduced. The image, the sound, and the seat are controlled accordingto the needs of the user. As described above, on the assumption that theside-window transparent display is divided into regions, the content,when dropping on to the first region {circle around (3)} or the secondregion {circle around (1)}, is reproduced on the first region {circlearound (3)} or the second region {circle around (1)}, respectively. Thecontent, when dropping in a manner that straddles the border between thefirst region {circle around (3)} and the second region {circle around(1)}, is reproduced on the entire screen (the first region {circlearound (3)} and the second region {circle around (1)}).

FIG. 54 is a view illustrating an example of control of the displaypresentation region when moving the content according to the presentembodiment of the present disclosure.

Occupant A watches the content using the front-side display. Occupant Cand Occupant B are allocated regions, respectively, of the side-windowdisplay and check the additional information (AR information) on theexternal object of interest.

When Occupant A transfers the image content to Occupant B, Occupant Awatches the same image content using the front-side display, andOccupant B watches the same image content on the allocated region of theside-window display.

Occupant B may make a request for a function of expanding a videoreproduction region of the side-window display. At this point, OccupantB may make a request to expand a content watching region for Occupant Bto a region allocated to Occupant C, through gazing recognition, gesturerecognition, or the like. Occupant C may check the additionalinformation (AR information) on the external object of interest, using areallocated region of the display that results from the expansion.

FIG. 55 is a flowchart illustrating a method of controlling a vehicledisplay according to still another embodiment of the present disclosure.

The method of controlling a vehicle display according to the presentembodiment of the present disclosure includes Step S5510 of receivingcontent movement input, Step S5520 of generating a command for contentmovement control, and Step S5530 of moving a content presentation regionand performing content display according to the command.

In Step S5510, a request for movement of a display region for thecontent is received through at least one of the gesture of the occupantinside of the vehicle, the voice thereof, the inputting through thebutton on the screen, and the inputting through the smartphone.

In Step S5520, the command for the content movement control is generatedin such a manner that the reproduction region is set according to thevideo arrangement position.

In Step S5530, by moving the content presentation region, soundreproduction is controlled, and a direction of the seat is changed.

FIG. 56 is a block diagram illustrating a system for controlling avehicle display according to still another embodiment of the presentdisclosure.

The system for controlling a vehicle display according to the presentembodiment of the present disclosure includes a gaze recognition unit5610 that acquires gaze information on the display region for theoccupant inside of the vehicle, a control unit 5620 that controls arequest for and acceptance of an initiative to occupy the displayaccording to the gaze information, and a display unit 5630 that displaysthe content according to whether or not the initiative is requested andaccepted.

When, as a result of the recognizing by the gaze recognition unit 5610,it is checked that a first occupant does not gaze at a screen allocatedto himself/herself and that a second occupant makes a request for theinitiative while gazing at the screen allocated to the first occupant,the control unit 5620 performs control in such a manner that aninitiative for the screen allocated to the first occupant is assigned tothe second occupant.

In a case where, as a result of the recognizing by the gaze recognitionunit 5610, the first occupant gazes at the screen allocated tohimself/herself and the second occupant makes a request for theinitiative while gazing at the screen allocated to the first occupant,when it is determined that a request for the initiative is notacceptable, the control unit 5620 performs control in such a manner thata notification of the request for the initiative is not transmitted tothe first occupant.

In a case where, as a result of the recognizing by the gaze recognitionunit 5610, the first occupant gazes at the screen allocated tohimself/herself and the second occupant makes a request for theinitiative while gazing at the screen allocated to the first occupant,when it is determined that the request for the initiative is acceptable,the control unit 5620 transmits the notification of the request for theinitiative to the first occupant. Moreover, in a case where the firstoccupant accepts the request for the initiative, the control unit 5620transmits to the display unit 5630 a control command for minimizing afunction widget that is being used by the first occupant.

FIGS. 57A to 57C are views illustrating an example of display occupationcontrol in accordance with a result of gaze monitoring according to thepresent embodiment of the present disclosure.

With reference to FIG. 57A, the vehicle display is divided into a presetnumber of regions (for example, two regions). The preset number ofregions are allocated to occupants (for example, Occupant A and OccupantB), respectively. Occupant A listens to music using the region allocatedto himself/herself, and a screen for music reproduction is displayed onthe allocated region. Occupant B watches a video using the regionallocated to himself/herself, and a screen for video reproduction isdisplayed on the allocated region. It is assumed that music is played ona display region for Occupant A and that, as a result of acquiring animage captured by an eye tracking camera, Occupant A does not gaze atthe display region. At this point, whereas Occupant A may not make arequest to Occupant B for the initiative, Occupant B may make a requestto Occupant A for the initiative.

With reference to 57B, Occupant B gazes at the screen allocated toOccupant A for a preset time or longer or makes a request for theinitiative for the screen allocated to Occupant A by making a specificgesture. With reference to FIG. 57C, it is possible for Occupant B tooperate a screen operation on an entire vehicle display region byacquiring the initiative for the screen allocated to Occupant A.

FIGS. 58A and 58B are views illustrating a situation of control forrejection of the request for the initiative that is in accordance withthe result of the gaze monitoring according to the present embodiment ofthe present disclosure.

With reference to FIG. 58A, the vehicle display is divided into a presetnumber of regions (for example, 2 regions). The preset number of regionsare allocated to occupants (for example, Occupant A and Occupant B),respectively. Occupant A listens to music using the region allocated tohimself/herself, and the screen for music reproduction is displayed onthe allocated region. Occupant B watches a video using the regionallocated to himself/herself, and the screen for video reproduction isdisplayed on the allocated region. A situation is assumed where, as aresult of acquiring the image captured by the eye tracking camera,Occupant A and Occupant B gaze at the display regions, respectively,that are allocated to themselves (Occupant A searches for a song, usingthe gaze recognition function, and Occupant B is watching a video thatis being reproduced on the display region allocated to himself/herself).

With reference to FIG. 58B, although Occupant B gazes at the screenallocated to Occupant A for a preset time or longer or makes a specificgesture, a state where an initiative button is not highlighted and wherethe request for the initiative is not acceptable is entered. That is,Occupants A and B can only reproduce content on the regions,respectively, that are allocated to themselves and cannot make a requestfor the initiative for the region to each other.

FIGS. 59A to 59C are views illustrating a situation of control for therequest for and the acceptance of the initiative that are in accordancewith the result of the gaze monitoring according to the presentembodiment of the present disclosure.

With reference to FIG. 59A, the vehicle display is divided into a presetnumber of regions (for example, 2 regions). The preset number of regionsare allocated to occupants (for example, Occupant A and Occupant B),respectively. Occupant A listens to music using the region allocated tohimself/herself, and the screen for music reproduction is displayed onthe allocated region. Occupant B watches a video using the regionallocated to himself/herself, and the screen for video reproduction isdisplayed on the allocated region. A situation is assumed where, as aresult of acquiring the image captured by the eye tracking camera,Occupant A and Occupant B gaze at the display regions, respectively,that are allocated to themselves (Occupant A searches for a song, usingthe gaze recognition function, and Occupant B is watching a video thatis being reproduced on the display region allocated to himself/herself).

With reference to 59B, it is assumed that video content takes precedenceover music content in making a request for an initiative for a region.Therefore, in a case where Occupant B gazes at the screen allocated toOccupant A for a preset time or longer or makes a specific gesture, apop-up message for the request for the initiative by Occupant B isdisplayed.

With reference to FIG. 59C, when Occupant A accepts the request for theinitiative by Occupant B, a music widget used by Occupant A isminimized, and it is possible for Occupant A to continuously search fora song. It is possible for Occupant B to use the display regionallocated to himself/herself as a region for wearing makeup and to watcha video on a large-sized video reproduction region, using the displayregion previously allocated to Occupant A.

FIG. 60 is a flowchart illustrating a method of controlling a vehicledisplay according to still another embodiment of the present disclosure.

The method of controlling a vehicle display according to the presentembodiment of the present disclosure includes Step S6010 of recognizinga gaze of the occupant, Step S6020 of controlling an initiative for adivisional display region according to a result of recognizing the gaze,and Step S6030 of controlling display occupation according to setting ofthe initiative.

in a case where it is checked in Step S6010 that the first occupant doesnot gaze at the screen allocated to himself/herself and that the secondoccupant gazes at the screen allocated to the first occupant, in StepS6020, the initiative is controlled in such a manner that the screenallocated to the first occupant is reallocated to the second occupant.

In a case where it is checked in Step S6010 that the first occupantgazes at the screen allocated to himself/herself and that the secondoccupant gazes at the screen allocated to the first occupant, thesituation is that the request for the initiative is not acceptable.Therefore, in Step S6020, control is performed in such a manner that thenotification of the request for the initiative is not transmitted.

In a case where it is checked in Step S6010 that the first occupant doesnot gaze at the screen allocated to himself/herself and that the secondoccupant gazes at the screen allocated to the first occupant, thesituation is that the request for the initiative is acceptable.Therefore, in Step S6020, the notification of the request for theinitiative by the second occupant is provided to the first occupant. Ina case where the first occupant allows the assigning of the initiative,in Step S6030, display occupation control is performed in such a mannerthat the function widget that is being used by the first occupant isminimized.

FIG. 61 is a block diagram illustrating a system for controlling avehicle display according to still another embodiment of the presentdisclosure.

The system for controlling a vehicle display according to the presentembodiment of the present disclosure includes an operation recognitionunit 6110 that recognizes a user request for a widget movement, a widgetmovement control unit 6120 that generates a control command for a widgetmovement on a region of the vehicle display at the request of the user,and a display unit 6130 that displays a widget according to the controlcommand for the widget movement.

The operation recognition unit 6110 recognizes the user request using atleast one of a touch input signal, a gesture input signal, and the gazeinformation.

When scrolling is performed with a widget touch, the widget movementcontrol unit 6120 generates a control command in such a manner that awidget group is moved among a plurality of regions of the vehicledisplay.

The widget movement control unit 6120 allocates regions to a pluralityof occupants, respectively, according to their locations, and generatesthe control command for the widget movement at the request of eachoccupant.

When the scrolling is performed on a first region, the widget movementcontrol unit 6120 generates a control command in such a manner that thescrolling is operative only on the corresponding widget drawer on thefirst region and that a widget drawer is hidden from view on a secondregion.

The widget movement control unit 6120 determines whether or not the usergazes at a widget and generates the control command for highlighting thecorresponding widget.

In a case where a widget is dragged and held onto a preset end portionof the first region, the widget movement control unit 6120 generates acontrol command for moving a widget group on a lower end portion of thefirst region among the regions.

FIGS. 62A and 62B are views illustrating a process of operating a screenaccording to recognition of a plurality of users according to thepresent embodiment of the present disclosure.

With reference to FIGS. 62A and 62B, a side display is divided into aplurality of regions (for example, 2 regions). User A has the right tocontrol a plurality of regions. First to fourth widgets are arranged ona lower end portion of a first region (a left-side region), and fifth toeighth widgets are arranged on a lower end portion of a second region (aright-side region).

When User A performs scrolling by applying a touch signal to a lower endportion of the display region, a group of 4 widgets is moved on aper-group basis among the plurality of regions of the display. That is,group are moved in such a manner that the first to fourth widgets arearranged on the second region and that the fifth to eighth widgets arearranged on the first region. These widgets are aligned by beingmagnetically snapped.

In a case where User A assigned the first region is using the firstregion and where User B assigned the second region is using the secondregion, User A who is relatively close to the side display performswidget scrolling using a screen touch. For remote control, User B who isrelatively remoter from the side display performs the widget scrollingusing at least one of a gazing recognition technique, a gesturerecognition technique, and a technique of recognition through operationof the smartphone. For example, through the recognition of the gaze orgesture of User B, scrolling occurs in such a manner that a change fromthe first, second, third, and fourth widgets to the fifth, sixth,seventh, and eighth widgets occurs on the lower end portion of thesecond region for User B.

FIG. 63 is a view illustrating a process of personalization operationthrough display separation according to the present embodiment of thepresent disclosure.

It is assumed that the first region and the second region are allocatedto User A and that the first to fourth widgets are arranged on a lowerend portion of the first region and the fifth to eighth widgets arearranged on the lower end portion of the second region. At this point,in a case where a gaze of a different user (User B) is recognized, awidget drawer is provided on a per-user basis. At this point, one of thefirst region and the second region is reallocated, considering a ridingposition of the occupant and a gaze direction of the occupant. FIG. 63illustrates that the first region and the second region are reallocatedto User A and User B, respectively. When scrolling is performed on thefirst region (Display A), the scrolling is operative only on thecorresponding drawer, and a drawer is hidden from view on the secondregion (Display B).

FIGS. 64A and 64B are views illustrating a process in which a user at aremote location operates the screen according to the present embodimentof the present disclosure.

A situation is assumed where, among divisional regions of the sidedisplay, the first region and the second region are allocated to User Aand User B, respectively.

When it is recognized that User B gazes at the lower end portion of thedisplay (widget eye gaze), a widget (Widget 3) corresponding to the gazedirection is highlighted. Subsequently, in a case where User B selects awidget region using a gesture (grab) and makes a request for movement inthe leftward-rightward direction, a widget scrolling operation isperformed. In addition, in a case where User B makes a pinch gesturewhile gazing at a specific widget, this pinch gesture is recognized asan operation of selecting the corresponding widget. In a case where thecorresponding widget (Widget 7) is selected, the corresponding widget isdisplayed on an upper end portion of the second region in an enlargedmanner.

FIG. 65 is a view illustrating a process of moving an individual widgetusing the touch input signal according to the present embodiment of thepresent disclosure.

In a case where Widget 2 is long-pressed through touch inputting, arearrangement mode is entered. When the corresponding widget is draggedand dropped to a desired position, the order in which widgets arearranged is changed. For example, as illustrated in FIG. 65 , thereoccurs a change from a state where the first, second, third, and fourthwidgets are arranged in this order on the lower end portion of the firstregion described above to a state where the first, third, second, andfourth widgets are arranged in this order on the lower end portion ofthe first region described above.

FIGS. 66A and 66B are views illustrating a process of moving theindividual widget using the gaze recognition and the gesture accordingto the present embodiment of the present disclosure.

When an occupant who is relatively remote from the side display gazes atthe lower end portion of the display, a widget on a region correspondingto the corresponding gaze direction is displayed in a highlightedmanner. In a case where the user grabs the widget region using aspecific gesture and holds the widget region for a preset time (nseconds), the rearrangement mode is entered. It is possible to move thegrabbed widget in the leftward-rightward direction. The user may changethe order in which the widgets are arranged, by dragging and droppingthe grabbed widget to the desired position. That is, there may occur achange from the state where the first, second, third, and fourth widgetsare arranged in this order on the lower end portion of the first regiondescribed above to a state where the first, third, second, and fourthwidgets are arranged in this order on the lower end portion of the firstregion described above.

FIGS. 67A and 67B are views illustrating a process of moving the widgetgroup by moving a single widget according to the present embodiment ofthe present disclosure.

Like in the example described above, it is assumed that the first regionand the second region are allocated to User A and User B, respectively.The first, second, third, and fourth widgets are arranged in this orderon the lower end portion of the first region, the fifth, sixth, seventh,and eighth widgets are arranged in this order on the lower end portionof the second region.

In a case where User A drags and holds the widget (Widget 2) onto theend portion of the first region, the widget group on the lower endportion of the first region is moved. That is, a group including thefirst to fourth widgets is moved to the lower end portion of the secondregion, and a group including the fifth to eighth widgets is moved tothe lower end portion of the first region.

In addition, in a case where User A releases the corresponding widget(Widget 2) onto a desired position, the order in which the widgets arearranged is completely changed. Thus, the fifth, sixth, second, andseventh widgets are arranged in this order on the lower end portion ofthe first region, and the eighth, first, third, and fourth widgets arearranged in this order on the lower end portion of the second region.

When the left side of the second region is swiped to the left-sidewidget group, the widget group is moved among the regions of thedisplay. That is, the eighth, first, third, and fourth widgets arearranged in this order on the lower end portion of the first region, andthe fifth, sixth, second, and seventh widgets are arranged in this orderon the lower end portion of the second region.

in order to move one widget from one region of the display to theseparated region of the display, the one widget may be arranged at adesired position by being dragged and dropped. However, in a case wherethe screen is physically divided, when dragging from the first region tothe second region is performed, a phenomenon occurs where a touch doesnot occur between the first region and the second region. In order toeasily move the widget between the two separated regions of the display,in a case where the widget on the first region is dragged and heldfarthest in the direction in which the widget on the first region is tobe moved, the widget group on the lower end portion of the first regionis move and arranged at a desired position (the lower end portion of thesecond region). It is possible to move a group of a plurality of widgetsby moving one widget.

FIG. 68 is a flowchart illustrating a method of controlling a vehicledisplay according to still another embodiment of the present disclosure.

The method of controlling a vehicle display according to the presentembodiment of the present disclosure includes Step S6810 of recognizinga user motion, Step S6820 of generating the control command for thewidget movement, and Step S6830 of controlling the display according toa widget movement.

In Step S6810, the user motion is recognized using at least one of thetouch input signal, the gesture input signal, and the gaze information.

In Step S6820, when scrolling is performed, with the widget touch, on aplurality of regions into which the vehicle display region is divided, acontrol command is generated in such a manner that a widget group ismoved among the plurality of regions.

In Step S6820, the control command for the widget movement withinregions that are allocated according to positions, respectively, of theplurality of occupants is generated at the request of each occupant.

In Step S6820, when scrolling is performed on the first region, acontrol command is generated in such a manner that the scrolling isoperative only on the corresponding widget drawer, and a control commandis generated in such a manner that the widget drawer is hidden from viewon the second region.

In Step S6820, it is determined whether or not the user gazes at awidget, and a control command for highlighting the corresponding widgetis generated.

In Step S6820, in the case where a widget is dragged and held onto apreset end portion of the first region, a control command for moving awidget group on the lower end portion of the first region among theregions is generated.

FIG. 69 is a block diagram illustrating a system for controllingreproduction of content on a medium according to still anotherembodiment of the present disclosure.

The system for controlling reproduction of content on a medium accordingto the present embodiment of the present disclosure includes a controltarget recognition unit 6910 that recognizes target content on a mediumthat is to be controlled by the user, among content items on mediumsthat are reproduced using the vehicle display, a control commandgeneration unit 6920 that generates a control command according to aresult of recognizing a user request for the target content on themedium, and a medium control unit 6930 that controls the content on themedium according to the control command.

The control target recognition unit 6910 recognizes the target contenton the medium using at least one of gaze information of the user,control request information that is received from a device carried bythe user, and positional information of the device carried by the user.

The control command generation unit 6920 checks whether that the devicecarried by the user is positioned inside of or outside of the vehicle,using a UWB sensor, and accordingly generates a control command for thetarget content on the medium.

The control command generation unit 6920 recognizes the user requestsincluding a request for adjustment of the volume of sound from thetarget content on the medium and generates the control command.

The medium control unit 6930 adjusts the volume of sound from the targetcontent on the medium according to the control command.

FIG. 70 is a view illustrating a process of distinguishing andrecognizing operations for the content on the medium that usesmartphones of occupants inside of and outside of the vehicle accordingto the present embodiment of the present disclosure.

With reference to FIG. 70 , a first user group is watching first contenton a medium using the side-window display inside of the vehicle, and asecond user group is watching second content on the medium using thevehicle display outside of the vehicle.

Using the WiFi Direct technology, the display on which videocasting isperformed is selected from the smartphone carried by the user, and thevolume of sound from a speaker for the display on which the videocastingis performed is adjusted.

With reference to FIG. 70 , among regions of the side-window display andamong regions of the front-side display, a target region to becontrolled is selected from a smartphone application, and the volume ofsound from the content on the medium that is reproduced on the targetregion is adjusted. The display on which the content on the medium is tobe reproduced is selected, and the volume of sound from the content isadjusted through dragging.

FIG. 71 is a view illustrating a process of distinguishing andrecognizing the operations for the content on the medium, using gazerecognition information and information for the operations for thecontent on the medium that use the smartphones according to the presentembodiment of the present disclosure.

The eye tracking camera is arranged inside of and outside of thevehicle, and the gaze information of the user who gazes at theside-window display or the front-side display is acquired using the eyetracking camera.

The target content on the medium, the volume of sound from which theuser desires to adjust, is determined using the gaze information andinformation on functional operation of the smartphone. The volume ofsound from the content on the medium is adjusted according to volumeadjustment information that is input through the smartphone application.

When the menu item “eye gaze remote” is selected from the smartphoneapplication, the user adjusts the volume of sound through the menu item“swipe up and down” while gazing at the display region on which thecontent on the medium is being reproduced.

FIG. 72 is a view illustrating a process of distinguishing andrecognizing the operations for the content on the medium using the gazerecognition information and UWB communication according to the presentembodiment of the present disclosure.

The eye tracking camera is arranged inside of and outside of thevehicle, and the gaze information of the user who gazes at theside-window display or the front-side display is acquired using the eyetracking camera.

Through the UWB sensor, it is checked whether or not the smartphone ispositioned inside of the vehicle or outside of the vehicle.

The target content on the medium, the volume of sound from which theuser desires to adjust, is determined using the gaze information and aresult of sensing by the UWB sensor. The volume of sound from thecontent on the medium is adjusted according to the volume adjustmentinformation that results from operating a hardware sound volume buttonof the smartphone.

That is, in a case where the user adjusts the volume of sound byoperating the hardware sound volume button of the smartphone carried byhimself/herself while gazing at the display on which the content on themedium is being reproduced, it is recognized that the user makes arequest for the adjustment of the volume of sound from the content onthe medium that is being reproduced on the region at which the usergazes. Accordingly, the volume of sound is adjusted.

FIG. 73 is a flowchart illustrating a method of controlling reproductionof content on a medium according to still another embodiment of thepresent disclosure.

The method of controlling reproduction of content on a medium accordingto the present embodiment of the present disclosure includes Step S7310of acquiring information on the environment for controlling the contenton the medium, Step S7320 of generating a control command fordetermining control-target content on the medium using the informationon the environment for controlling the content on the medium, andgenerating a control command for the control-target content on themedium, and Step S7330 of performing control for reproducing the contenton the medium according to the control command.

In Step S7310, pieces of information on a plurality of content items onmediums that are reproduced on a plurality of display regions,respectively, in the vehicle, are checked.

In Step S7320, the control-target content on the medium is determinedusing at least one of the gaze information of the user, the controlrequest information received from a device carried by the user, and thepositional information of the device carried by the user.

In Step S7320, it is checked whether a request is for controlling thecontent on the medium that the occupant staying inside of the vehiclewatches or for controlling the content on the medium that the userstaying outside of the vehicle watches and thus the request isrecognized.

In Step S7330, control of the volume of sound from the content on themedium is performed on the basis of the control command according to therequest for the adjustment of the volume of sound from the content onthe medium.

FIG. 74 is a block diagram illustrating a system for controllingreproduction of content on a medium according to still anotherembodiment of the present disclosure.

The system for controlling reproduction of content on a medium accordingto the present embodiment of the present disclosure includes areproduction environment recognition unit 7410 that recognizes anenvironment for reproducing content items on a plurality of mediums thatare reproduced using the vehicle display, a control command generationunit 7420 that generates a control command for preventing soundinterference among the plurality of mediums, and a medium contentcontrol unit 7430 that controls the content on the medium according tothe control command.

The reproduction environment recognition unit 7410 recognizesinformation on a situation where the first user group watches firstcontent in a state where one or several of vehicle seats are detachedand are moved to the outside and information on a situation where thesecond user group watches second content in a state where one or severalof the vehicle seats are rotated.

In a situation where the first user group watches the first content thatis reproduced front side back through a front-side projection display,the control command generation unit 7420 receives a content controlrequest from a device corresponding to the first user group andaccordingly generates a control command.

In a situation where the second user group watches the second contentthat is reproduced through the side-window display, the control commandgeneration unit 7420 generates a control command in such a manner that asound bubble is formed using a headrest speaker.

FIG. 75 is a view illustrating a process of preventing the soundinterference according to the present embodiment of the presentdisclosure.

With reference to FIG. 75 , in a situation where the vehicle travels,the occupants inside of the vehicle watch content using the front-sideprojection display. One or several of the vehicle seats are detached andmove to the outside, and the other vehicle seats are rotated. The firstuser group watches the first content while sitting on the vehicle seatthat is detached and moved to the outside, and the second user groupwatches the second content while sitting on the rotated vehicle seat.

The first user group watches the first content through front-side-backreproduction on the front-side projection display (an external-theatermode). At this point, the first user group listens to sound from thefirst content through a surface vibration speaker. The first user groupcontrols the volume of sound from the first content and reproduction ofthe first content using the smartphone. The surface vibration speaker isused for communication warning. The surface vibration speaker reproducessound according to a direction that varies with the location of apedestrian and reproduces the sound toward a direction in which amovement out of a safety area takes place. The surface vibration speakergenerates an alarm sound in a direction in which an indicator light isturned on.

The second user group watches the second content using the side-windowdisplay while sitting on the rotated vehicle seat (an internal-theatermode). The second user group listens to the sound from the secondcontent using the headrest speaker. The headrest speaker forms the soundbubble and thus prevents interference with external sound. In addition,the headrest speaker blocks external noise by performing an active noisecanceling function.

FIG. 76 is a flowchart illustrating a method of controlling reproductionof content on a medium according to still another embodiment of thepresent disclosure.

The method of controlling reproduction of content on a medium accordingto the present embodiment of the present disclosure includes Step S7610of acquiring information on an environment for reproducing content itemson a plurality of mediums, Step S7620 of generating a control commandassociated with prevention of sound interference among the plurality ofmediums, and Step S7630 of performing control for reproducing thecontent on the medium according to the control command.

In Step S7610, the information on the situation where the first usergroup watches the first content in the state where one or several of thevehicle seats are detached and are moved to the outside and theinformation on the situation where the second user group watches thesecond content in the state where one or several of the vehicle seatsare rotated are acquired.

Step S7620, sound from the first content is reproduced through thesurface vibration speaker, and a control command for a control requestthat is received through the device corresponding to the first usergroup is generated.

In Step S7620, sound from the second content is generated using theheadrest speaker, and a control command for forming the sound bubble isgenerated.

FIG. 77 is a block diagram illustrating a system for determining a riskelement and providing a warning according to still another embodiment ofthe present disclosure.

The system for determining a risk element and providing a warningaccording to the present embodiment of the present disclosure includesan approaching-object recognition unit 7710 that recognizes whether ornot a non-registered object approaching the vehicle is present, acontrol command generation unit 7720 that determines whether or not arisk situation occurs due to the approaching by the non-registeredobject and controls a control command associated with providing awarning, and a notification providing unit 7730 that provides a warningnotification according to the control command.

Using at least one of an image sensor, a radar sensor, and a lidarsensor, the approaching-object recognition unit 7710 checks whether ornot the non-registered object is present.

The approaching-object recognition unit 7710 checks whether or not thenon-registered object is present, by checking whether or not an alreadyregistered key is retained.

The control command generation unit 7720 analyzes a behavioral patternof the non-registered object and determines whether or not the risksituation occurs.

Considering information on a situation where the vehicle is parked orstops, the control command generation unit 7720 determines whether ornot the risk situation occurs due to the approaching by thenon-registered object.

The control command generation unit 7720 checks whether or not theoccupant stays inside of the vehicle and provides a warning using atleast one of the speaker, the vehicle display, and a device of the userof the vehicle.

The system for determining a risk element and providing a warningaccording to the present embodiment of the present disclosure furtherincludes a leaving-object recognition unit 7740 that recognizes anobject moving out of a preset region of the vehicle. The control commandgeneration unit 7720 generates a control command in such a manner thatthe warning notification is provided to the object that moves out of apreset region.

FIGS. 78A to 78C are views illustrating a situation of providing awarning of an external risk element and a warning of a movement out of asafety area according to the present embodiment of the presentdisclosure.

With reference to FIGS. 78A to 78C, in a situation where a passengerrides in the vehicle, when it is determined that a person that is notregistered with the vehicle approaches the vehicle, a directionalwarning is provided using a speaker. In addition, screen displaying isperformed using the vehicle display in a direction from which thenon-registered person approaches the vehicle. A notification of asituation where the non-registered person approaches the vehicle isprovided.

At this point, the situation where the non-registered person approachesthe vehicle may be an actual risk element or may not be an actual riskelement in a situation where the non-registered person may pass by ormay approach the vehicle in order to ride in his/her vehicle parked in anearby parking space. Therefore, information on a situation wherevehicles are currently parked or stop (for example, a place, a timezone, and the presence or absence of a nearby vehicle) is analyzed, andit is determined whether or not to activate a process of providing awarning that the non-registered person approaches the vehicle. Forexample, in a case where, in a campsite on the bare ground, anothervehicle is not parked around the vehicle and where there is no sidewalk,when the non-registered person approaches the vehicle, the warning thatthe non-registered person approaches the vehicle is provided. Incontrast, in a situation where like in a parking lot in a shopping mall,another vehicle is parked around the vehicle and where many persons passby around the vehicle, the warning that the non-registered personapproaches the vehicle is not immediately provided. In this case, whenit is detected that the non-registered person sneaks around the vehicle,peeps into the vehicle, or touches his/her hand on the vehicle, it isdesirable that the warning is provided according to a result of thedetection.

In a situation where a passenger does not ride in the vehicle, in a casewhere the non-registered person approaches the vehicle and where this isdetermined as an actual risk element, the directional warning isprovided toward a direction from which the non-registered personapproaches the vehicle, and the warning is provided to thenon-registered person by performing screen displaying using the vehicledisplay. In addition, the warning is provided to a smart device of analready-registered user of the vehicle, and thus the user who staysoutside of the vehicle is alerted that the risk element occurs, that is,the non-registered person approaches the vehicle.

In the above-mentioned example, in order to determine the external riskelement, by distinguishing face IDs through an external camera, it isdetermined whether or not a person who approaches the vehicle is analready-registered user. The warning is provided to a person, having nodigital key or smart key, who approaches the vehicle. In a case wherethe non-registered person stays at a short distance away from thevehicle for a preset time, touches his/her hand on the vehicle, orapplies an impact to the vehicle, the warning is provided to thenon-registered person. A distance and a shape are measured andrecognized, respectively, through a radar or lidar sensor of thevehicle. Using the radar sensor, the presence or absence of an object,information on a distance to the object, information on a size of theobject are determined. Using the lidar sensor, the shape of the objectand a surrounding environment are recognized. Using a camera, asurrounding object is identified. The warning varies according towhether a warning target is a person or an animal. In a case where thewarning target is a person, a warning message is displayed on anexternal display, and a warning siren sound is provided. In a case wherethe warning target is an animal, an external light source is caused toflicker, and a warning sound that varies according to a kind of animalis provided. For example, a warning sound having a low frequency (150 to8000 Hz) is provided to a wild pig.

In the above-mentioned campsite on the bar ground, in a case where oneor several occupants stay inside of the vehicle and where an object (auser, a companion dog, or the like) that stays close to the vehiclemoves out of a preset area, a warning that the object moves out of asafety area is provided to the occupants who stay inside of the vehicle.At this point, a directional warning is provided toward a direction inwhich the object moves out of the safety area, and the warning messageis presented using the vehicle display. In order to determine whether ornot the object moves out of the preset area, a separation distance isdetermined by performing UWB communication with a smartphone carried bythe user, a smart tag attached to the companion dog, or the like. In acase where the user, the companion dog, or the like moves out of thepreset area, the warning is provided.

FIG. 79 is a view illustrating a method of determining a risk elementand providing a warning according to still another embodiment of thepresent disclosure.

The method of determining a risk element and providing a warningaccording to the present embodiment of the present disclosure includesStep S7910 of recognizing at least one of an approaching object and anobject moving out of an area that is preset with respect to the vehicle,Step S7920 of generating a control command for providing a warning ofoccurrence of a risk situation according to a result of the recognition,and Step S7930 of providing the warning notification according to thecontrol command.

In Step S7920, in a case where it is recognized that the objectapproaches the vehicle, considering a result of analyzing the behavioralpattern of the object that is not registered with the vehicle and theinformation on the situation where the vehicle is parked or stops, it isdetermined whether or not the risk situation occurs.

In Step S7920, in a case where it is recognized that the object movesaway from the vehicle, the object moving away from the vehicle isrecognized by acquiring information on the separation distance to adevice of the object moving away from the vehicle.

In Step S7930, the warning notification is provided using at least oneof the speaker, the vehicle display, and the device of the user of thevehicle.

FIG. 80 is a block diagram illustrating a vehicle control system forreducing motion sickness according to still another embodiment of thepresent disclosure.

The vehicle control system for reducing motion sickness according to thepresent embodiment of the present disclosure includes a motion-sicknessreduction control request unit 8010 that makes a request formotion-sickness reduction control according to at least one of a resultof analyzing bio-information of the user and a request of the user, acontrol command generation unit 8020 that generates a control commandfor executing a logic for the motion-sickness reduction control, and adisplay and in-vehicle layout change unit 8030 that, according to thecontrol command, determines a display region on which content isreproduced and changes a layout of an in-vehicle seat.

In a case where the bio-information is analyzed, the motion-sicknessreduction control request unit 8010 acquires the bio-information usingat least one of the camera and the radar, analyzes the acquiredbio-information, and makes a request for the motion-sickness reductioncontrol.

The control command generation unit 8020 generates a control command forperforming seat reclining in a motion-sickness reduction mode.

The control command generation unit 8020 generates a control command foradjusting an angle of view of a projector according to an angle by whichthe seat is reclined and for changing the region of the display on whichthe content is reproduced.

FIGS. 81A to 81C are views illustrating that the display and thein-vehicle layout are changed for motion-sickness reduction according tothe present embodiment of the present disclosure.

Content is reproduced using the projection display (a front-side glasspane). A heart rate and a respiration rate are measured on the basis ofan image of the inside of the vehicle that is acquired by the camera andinformation acquired by a wireless radar, and thus a bio-signal isdetected.

The logic for the motion-sickness reduction control is executed when, asa result of detecting the bio-signal, it is checked that there is alikelihood that motion sickness will occur, or at the request of theuser. With the logic for the motion-sickness reduction control, the seatoperates in a gravity-free seat mode. The reproduction region is changedin such a manner that content is reproduced using the projection display(a roof glass pane). A sound for the motion-sickness reduction isreproduced through the headrest speaker. The active noise cancelingfunction is performed, and thus a seat-to-seat conversation function isprovided. As the sound that is provided to the user, an external soundis reproduced in such a manner that the user can feel a vehicle speed,or a sound associated with relaxation or a sound of nature isreproduced. A channel in which the sound is reproduced varies accordingto the a situation where the vehicle travels. For example, when thevehicle travels in a straight line, the sound is reproduced in twochannels, that is, left and right channels. When the vehicle turns tothe left, the sound in the left channel is reproduced, and when thevehicle turns to the right, the sound in the right channel isreproduced.

FIG. 82 is a view illustrating a variable display layout of a front-sidedisplay according to the present embodiment of the present disclosure.

A communication lamp region is arranged on an upper end portion of thefront-side projection display, and a cluster and task bar region isarranged on a lower end portion thereof. The external environment isshown through the center portion thereof (the transparent display).

The communication lamp region may be arranged on the upper end portionof the front-side projection display, the cluster and task bar regionmay be arranged on the lower end portion thereof, and a full-size widgetscreen may be arranged on the center portion thereof.

A path information provision screen may be arranged on the upper endportion of the front-side projection display, and the cluster and taskbar region may be arranged on the lower end portion thereof. Moreover,the center portion thereof are divided into two parts, and a half-sizewidget screen and a transparent display region that is seen from theoutside may be arranged on the two parts, respectively.

The cluster and task bar region may be arranged on the lower end portionof the front-side projection display, and the video content may bereproduced in a full screen mode on the remaining region thereof.

The video content may be reproduced on a roof glass pane projectiondisplay, and one portion of the video content may also be displayed on afront-side projection display region. The cluster and task bar regionmay be arranged on the lower end portion of the front-side projectiondisplay, an external environment may be seen through an upper end regionof the front-side projection display, and the external environment maybe seen through regions of the front-side projection display other thana lower end region thereof.

FIGS. 83A and 83B are views illustrating the projection display (afront-side glass pane) according to the present embodiment of thepresent disclosure.

The communication lamp region on which a text message that is providedto another vehicle or a pedestrian is displayed is arranged on an upperend region of the front-side projection display. Content information,traveling information, and the like are displayed on regions of thefront-side projection display other than the upper end region thereof insuch a manner that the occupant inside of the vehicle may check thecontent information, the traveling information, and the like. The angleof view of the projector is adjusted, and an image watching region and aviewing angle are adjusted.

FIG. 84 is a view illustrating a projector in a situation where theprojection display (the front-side glass pane) according to the presentembodiment of the present disclosure is used.

A rotational shaft of an actuator 8410 is connected to a rotationalshaft of a support structure 8420, and a projector 8430 is supported bya projector-cradling bracket structure 8440.

FIGS. 85A and 85B are views illustrating the projection display (theroof glass pane) according to the present embodiment of the presentdisclosure.

As described above, the layout of the in-vehicle seat is changed to themotion-sickness reduction mode, and the sound for the motion-sicknessreduction is provided through the headrest speaker.

The angle of view of the projector is adjusted, and content isreproduced on a region of the roof glass pane projection display. Theseat is reclined, and the viewing angle is adjusted in such a mannerthat the content may be watched through the roof glass pane projectiondisplay.

FIG. 86 is a view illustrating the projector in a situation where theprojection display (the roof glass pane) according to the presentembodiment of the present disclosure is used.

Rotation of the actuator 8410 rotates the projector 8430, and thus theangle of view is changed.

FIGS. 87A to 87G are views illustrating a process of moving an imageaccording to the present embodiment of the present disclosure.

A video is reproduced through the front-side projection display, and acommunication message is displayed, toward the outside, on an upper endregion of the front-side projection display.

At the request of the user, a region of an external communication lampis maintained, and a video is reproduced on a lower end region of theexternal communication lamp in a manner that is fitted to the full sizeof a screen.

When a function of reducing motion-sickness is performed, in a statewhere the region of the external communication lamp is maintained, avideo is moved to the roof glass pane projection display and isdisplayed thereon, and a seat reclining operation is performed.

The video is reproduced on a region of the roof glass pane projectiondisplay, and the external communication lamp operates concurrently.

While the video is being reproduced on the region of the roof glass paneprojection display, a GUI animation is played on a film region of alower end portion of the roof glass pane projection display in order forthe user to feel a moving speed of the vehicle. A line animation may bereproduced as if the vehicle were passing between two lines oflampposts. In addition, a sound of which a speed corresponds to a speedof the vehicle, a sound of nature, or like may be reproduced through theheadrest speaker.

While a video is being reproduced on the region of the roof glass paneprojection display, an external traveling situation is made to berecognizable with the naked eye by making the external communicationlamp region transparent. Thus, the sensation of the speed of the vehicleis possibly made to be experienced.

While a video is being reproduced on the region of the roof glass paneprojection display, the external traveling situation is made to berecognizable with the naked eye by making the entire front-sideprojection display transparent. Thus, the sensation of the speed of thevehicle is possibly made to be experienced.

FIG. 88 is a flowchart illustrating a vehicle control method forreducing motion-sickness according to still another embodiment of thepresent disclosure.

The vehicle control method for reducing motion-sickness according to thepresent embodiment of the present disclosure includes Step S8810 ofreceiving a request for the motion-sickness reduction control using atleast one of a result of analysis of the bio-information of the user andrequest information of the user, Step S8820 of generating a controlcommand for the motion-sickness reduction control, and Step S8830 ofchanging a setting of the display and the in-vehicle layout according tothe control command.

In Step S8810, in a case where the result of the analysis of thebio-information is used, the request for the motion-sickness reductioncontrol is received using the result of acquiring and analyzing thebio-information associated with motion-sickness.

In Step S8820, a control command for performing the seat reclining isgenerated.

In Step S8820, a control command for reproducing the sound for themotion-sickness reduction is generated.

In Step S8820, a control command for adjusting the angle of view of theprojector and changing the display region on which content is reproducedis generated.

In Step S8830, according to the control command, the display region onwhich the content is reproduced changed from the front-side projectiondisplay to the roof glass pane projection display, and a seat layout ischanged by adjusting a reclining angle.

FIG. 89 is a block diagram illustrating a system for controlling avehicle display according to still another embodiment of the presentdisclosure.

The system for controlling a vehicle display according to the presentembodiment of the present disclosure includes a content movement requestreception unit 8910 that receives the request for the movement of thecontent among the plurality of display regions, a control commandgeneration unit 8920 that generates a control command for moving thecontent according to the request for movement of content, changing thespeaker, and changing the in-vehicle seat layout, and a sound andin-vehicle layout changing unit 8930 that changes a sound and anin-vehicle seat layout according to the control command.

The content movement request reception unit 8910 receives at least oneof the gesture of the vehicle occupant, the inputting through the buttonon the screen, and the request for movement of an image using a deviceof the occupant.

According to the request for the movement of the content from a firstdisplay to a second display, the control command generation unit 8920generates the control command for display movement in such a manner thata separation distance between the first display and the second displayis decreased.

The control command generation unit 8920 performs grouping of speakergroups that reproduce a sound, considering a direction in which thecontent is moved.

The control command generation unit 8920 generates the control commandfor changing the in-vehicle layout and rotating the seat, consideringthe direction in which the content is moved.

FIGS. 90A to 90D are views illustrating a process of changing the soundand the in-vehicle layout according to movement of the image between thedisplays according to the present embodiment of the present disclosure.

1. In a situation where the front-side projection image is watched, theoccupant inside of the vehicle watches content using the front-sideprojection display. At this point, a sound is reproduced using a firstspeak group (for example, two speakers positioned in the front of theinside of the vehicle).

2. When an image is moved and reproduced, in a situation where theside-window display is moved, the request for the movement of thecontent between the plurality of display regions is received. At thistime, a request for movement of an image is received using the gestureof the vehicle occupant, or by performing inputting through the buttonon the screen or inputting through the smartphone. When the image ismoved, in order to minimize a distance between a pre-movement region(the front-side projection display) and a post-movement region (theside-window display), the side-window display that is a transparentdisplay is slid toward the front-side projection display.

3. In a situation where the image is to be moved, image content that isbeing watched is moved from a region of the front-side projectiondisplay to a region of the side-window display, and according to aresult of the movement, a seat of the vehicle occupant is rotated. Inaddition, a sound is reproduced through a second speaker group (withreference to FIG. 90 , content is moved to a region of a leftside-window display, and a speaker arranged on the left side of the rearof the vehicle is added to the first speaker group, thereby constitutingthe second speaker group).

4. In a situation where the side-window transparent display iscompletely moved, the image content is completely moved to the region ofthe side-window display, and the seat on which the occupant sits isrotated by 90 degrees in the leftward direction. Thus, a seat layout iscompletely changed. A stereo sound is reproduced using left and rightspeakers with respect to the side-window display.

FIG. 91 is a flowchart illustrating a method of controlling a vehicledisplay according to still another embodiment of the present disclosure.

The method of controlling a vehicle display according to the presentembodiment of the present disclosure includes Step S9110 of receivingthe request for the movement of the content among the plurality ofdisplay regions, Step S9120 of generating a control command associatedwith the movement of the content, the rotation of the seat, and controlof the speaker according to the request, and Step of S9130 of moving thecontent, rotating the seat, changing the in-vehicle layout, andgenerating controlling a speaker that reproduces a sound, according tothe control command.

In Step S9110, the request is received using at least one of the gestureof the vehicle occupant, the inputting through the button, and requestinformation that is input through an application on a device of thevehicle occupant.

In Step S9120, considering a direction in which the content is moved,the control command for controlling the movement of the display isgenerated.

In Step S9120, considering the direction in which the content is moved,the control command for determining the speaker group through which thesound is reproduced is generated.

FIG. 92 is a block diagram illustrating a system for controlling avehicle display according to still another embodiment of the presentdisclosure.

The system for controlling a vehicle display according to the presentembodiment of the present disclosure includes a display movement requestreception unit 9210 that receives a request for movement of a firstdisplay, a control command generation unit 9220 that generates a controlcommand for moving the first display according to the request, and anadditional information display unit 9230 that displays the additionalinformation on the first display moved according to the control commandand changes displaying of content presented on a region to which thefirst display is completely moved.

The display movement request reception unit 9210 receives the requestfor the movement of the first display using at least one of the gestureof the vehicle occupant, the inputting through the button, and therequest information that is received from the device of the vehicleoccupant.

The control command generation unit 9220 transmits the control commandfor moving along a moving rail the first display that is the transparentdisplay.

The additional information display unit 9230 makes transparent aprojection region, on which the content is presented, of the region towhich the first display is completely moved, and displays the additionalinformation, with the AR information corresponding to a position of anexternal object.

The additional information display unit 9230 adjusts a size of thecontent that is presented on the region to which the first display iscompletely moved, and displays the content of which the size isadjusted.

FIGS. 93A to 93E are views illustrating a sliding display according tothe present embodiment of the present disclosure.

With reference to FIG. 93A, the side-window transparent display isdivided into a plurality of regions. Among the plurality of regions, afirst transparent display 9310 is selected as a moving-target display.At this point, selection information on the moving-target display isreceived using at least one of the gesture of the occupant inside of thevehicle, the inputting through the button, gazing information, andoperational information of the device of the occupant inside of thevehicle.

With reference to FIG. 93B, the first transparent display 9310 is movedalong a moving rail to a left region of the front of the vehicle.

With reference to FIGS. 93C to 93E, the first transparent display 9310is moved along the moving rail and is arranged on a windshield region ofthe front of the vehicle.

FIGS. 94A to 94C are views illustrating a process of moving the displayand displaying AR content according to the present embodiment of thepresent disclosure.

FIG. 94A illustrates a situation where the first transparent display ismoved, but is not arranged on the windshield region of the front of thevehicle. The vehicle occupant watches path summarization information,navigation information, and medium information through the front-sideprojection display. At this point, the front-side projection display isvariable in a transparent mode and a non-transparent mode, but cannotexpress information on a situation in front of the vehicle, in an ARformat.

With reference to FIGS. 94B and 94C, the first transparent display ismoved from the side of the vehicle to the front of the vehicle. A regionof the front-side projection display, on which content (navigationinformation) is displayed is made to be transparent, and the additionalinformation (the AR information) is displayed on the transparent contentregion through the first transparent display. That is, one region of thefront-side projection display is made to be transparent, and navigationcontent is displayed as the AR content. Thus, it is possible that thevehicle occupant checks information through a more advancedsurrounding-environment information display. With reference to FIG. 94C,the navigation information is displayed as the AR content on an actualroad, and the additional information (coupon discount information, storeinformation, or the like) on an object in which the user is interested(an object at which the user gazes) is displayed as the AR content. In acase where a region of the front-side display on which information ispresented before the first transparent display is moved to the front ofthe vehicle overlaps a region of the transparent display, an amount ofthe front-side display information is adjusted for the vehicle occupant(an occupant on the right side of the first row), and thus thefront-side display information is displayed without information lossusing the front-side projection display.

FIG. 95 is a flowchart illustrating a method of controlling a vehicledisplay according to still another embodiment of the present disclosure.

The method of controlling a vehicle display according to the presentembodiment of the present disclosure includes Step S9510 of receivingthe request for the movement of the first display, Step S9520 ofgenerating the control command for moving the first display, and StepS9530 of moving the first display according to the control command anddisplaying the additional information on the region to which the firstdisplay is completely moved.

In Step S9520, the control command for moving along the moving rail thefirst display that is the transparent display is generated.

In Step S9530, the projection region, on which the content is presented,of the region to which the first display is completely moved is made tobe transparent, and the additional information is displayed, with the ARinformation corresponding to an external object.

In Step S9530, according to a state where the projection region and theregion of the first display overlap each other, an amount of contentthat is presented on the projection region is adjusted.

FIG. 96 is a block diagram illustrating a system for warping an image ona vehicle display according to still another embodiment of the presentdisclosure.

The system for warping an image on a vehicle display according to thepresent embodiment of the present disclosure includes an occupantmonitoring unit 9610 that checks information on a result of monitoringan occupant, a control command generation unit 9620 that generates acontrol command for warping an image according to the result ofmonitoring the occupant, and a warped image display unit 9630 thatdisplays a warped image according to the control command.

The occupant monitoring unit 9610 checks at least one of a position ofthe user who rides in the vehicle, information on a separation distancebetween a reference point of the vehicle display and the user, andgazing position information of the occupant.

The control command generation unit 9620 generates the control commandfor determining the degree of warping according to the result ofmonitoring the occupant.

The warped image is provided to provide a visual effect in which animage that may appear twisted according to the position of the occupantappears in the same manner as when viewed from the front. At this point,the degree of the warping of the image is determined by determiningpositions of the occupant's eyes and measuring a distance to the centralreference point of the display and an angle with respect thereof.Distances in the forward, backward, upward, and downward directions (aperson's sitting height or like) to the central reference point of thedisplay are all three-dimensionally measured. The warping of the imageis performed to provide the visual effect in which an image appears fromany angle in the same manner as when viewed from the front. Anin-vehicle camera (for example, a ToF or RGB camera) is used todetermine the distances. The measured distances are compared withreferences for measuring positions of a person's eyes and the person'shead and posture. At this point, values of points are used to calculatevalues of length and thus a trigonometric ratio or the like. In a casewhere the position of the occupant is moved, warping may be performed inreal time. Alternatively, after the position thereof is completelymoved, the degree of the warping may be set.

FIGS. 97A and 97B are views illustrating an example of an image based oneach occupant's position and an example of warping according to thepresent embodiment of the present disclosure.

With reference to FIGS. 97A and 97B, the image content is displayedthrough the side-window transparent display of the vehicle. At thispoint, it is assumed that a first user sits relatively close to theside-window transparent display and that a second user sits relativelyremote therefrom.

When it is assumed that the first user and the second gaze at the sameside-window transparent display, the degree of the warping is set tovary on the same display region.

That is, a different degree of warping is set for the first user whosits relatively close to the display than for the second user who sitsrelatively remote from the display.

From a front view of the display in FIG. 97 , in a case where theside-window transparent display is viewed from the front, it can be seenthat an image, having an actual degree of warping, that is provided tothe first user and an image, having an actual degree of warping, that isprovided to the second user are different in shape from each other.

FIG. 98 is a view illustrating the warping of the image according to thegaze information of the occupant according to the present embodiment ofthe present disclosure.

With reference to FIG. 98 , the image content is displayed through theside-window transparent display of the vehicle. At this point, the firstuser having a relatively small sitting height gazes at the side-windowtransparent display at a different angle than the second user having arelatively great sitting height. When it is assumed that the first userand the second gaze at the same side-window transparent display, thedegree of the warping is set to vary on the same display region.

That is, a different degree of warping is set for the first user havinga relatively small sitting height than for the second user having arelatively great sitting height.

From the front view of the display in FIG. 98 , in the case where theside-window transparent display is viewed from the front, it can be seenthat an image, having an actual degree of warping, that is provided tothe first user and an image, having an actual degree of warping, that isprovided to the second user are different in shape from each other.

FIG. 99 is a view illustrating the warping of the image according to adistance to the occupant and the gaze information according to thepresent embodiment of the present disclosure.

It is assumed that the seat within the vehicle is rotated and where theimage content is watched through the side-window display.

From FIG. 99 , in a case where the occupant who sits at the sittingposition watches the image content through the side-window display, itcan be seen that the image content on the display region appears in theshape of a rectangle to the occupant's eyes.

From the front view of the display that illustrates a warping situation,it can be seen that a height of one end of the warped image that isclose to the occupant is relatively smaller than a height of the otherend thereof that is relatively remote from the occupant.

FIG. 100 is a view illustrating a method of warping an image on avehicle display according to still another embodiment of the presentdisclosure.

The method of warping an image on a vehicle display according to thepresent embodiment of the present disclosure includes Step S10010 ofperforming monitoring of the occupant, Step S10020 of generating acontrol command for warping an image according to a result of monitoringof the occupant, and Step S10030 of displaying a warped image accordingto the control command.

In Step S10010, at least one of the riding position of the occupant whorides in the vehicle, the information on the separation distance of theuser to the reference point of the vehicle display, and the gazingposition information of the occupant is checked. In Step S10020, thecontrol command for determining the degree of the warping according to aresult of the checking in Step S10010 is generated.

In Step S10030, the warped image, in which the height of one end thereofthat is close to the occupant is set to be smaller than the height ofthe other end thereof that is relatively remote from the occupant, isdisplayed.

FIG. 101 is a block diagram illustrating a system for controlling avehicle display according to still another embodiment of the presentdisclosure.

The system for controlling a vehicle display according to the presentembodiment of the present disclosure includes an occupant monitoringunit 10110 that checks the information on the separation distancebetween the reference point of the vehicle display and the occupant, acontrol command generation unit 10120 that generates a control commandfor dividing the vehicle display into regions using the information onthe separation distance, and a region division display unit 10130 thatallocates the resulting regions of the display to the occupants,respectively, according to the control command, and displays content onthe resulting regions of the display according to the needs of theusers, respectively.

The occupant monitoring unit 10110 checks the information on theseparation distance using the in-vehicle camera.

The control command generation unit 10120 sets a value of a length ofthe display that is seen by each occupant's eyes, by determiningpositions of each occupant's eyes.

The control command generation unit 10120 generates the control commandfor adjusting the number of divisions of an image according to awidth-to-length ratio of the image that is provided to each occupant.

FIGS. 102A and 102B are views illustrating that the display is dividedinto regions according to the distance to the occupant according to thepresent embodiment of the present disclosure.

With reference to FIGS. 102A and 102B, Users A and B who ride in seats,respectively, in the second row check the additional information (the ARinformation) on the external object of interest using a left-sidetransparent display. At this point, contour-line information on theexternal object of interest is displayed as the additional information.Separation distances of User A and User B to the central reference pointof the left-side transparent display are different from each other. Theleft-side transparent display is divided into regions on the basis ofthe separation distances. That is, although there are two users, thetransparent display are not divided in half, that is, into two regions.The number of divisions of the screen is determined considering theinformation on the separation distance.

Accordingly, although sitting on different seats, users may use theregions, respectively, that result from dividing the display at the samedivision ratio of the image.

The in-vehicle camera (for example, the ToF or RGB camera) is used toacquire the information on the separation distance between thetransparent display and the user. Positions of the user's eyes aredetermined using a camera. The same value of the length of the displaythat is seen by the user's eyes is set not only by using a predeterminedposition of the display, but also by calculating the trigonometricratio.

That is, a divisional-display function is provided by determining thevalue of the length and thus adjusting an image region in such a mannerthat screen image ratios are the same at the same distance towarddirections, respectively, in which the occupants gaze at the display.

FIG. 103 is a flowchart illustrating a method of controlling a vehicledisplay according to still another embodiment of the present disclosure.

The method of controlling a vehicle display according to the presentembodiment of the present disclosure includes Step S10310 of performingmonitoring information of the occupant by checking the separationdistance between the vehicle display and the occupant, Step S10320 ofgenerating a control command for dividing the display into regions usingthe separation distance, and Step S10330 of providing regions thatresult from dividing the display according to the control command.

In Step S10310, the separation distance is checked using a cameracapturing an image of the inside of the vehicle.

In Step S10320, the value of the length of the display that is seen byeach occupant's eyes is set by determining the positions of eachoccupant's eyes, and the control command for adjusting the number ofdivisions of the image according to the width-to-length ratio of theimage is generated.

In Step S10330, the regions that result from dividing the display areprovided to the plurality of occupants, respectively, in an allocatedmanner, and content is displayed on the corresponding regions accordingto the needs of the plurality of occupants, respectively.

The methods according to the embodiments of the present disclosure maybe realized on a computer system or may be recorded on a recordingmedium. The computer system may include at least one processor, amemory, a user input device, a data communication bus, a user outputdevice, and a storage device. The constituent elements described aboveperform data communication with each other through the datacommunication bus.

The computer system may further include a network interface coupled to anetwork. The processor may be a central processing unit (CPU) or asemiconductor device that processes a command stored in the memoryand/or the storage device.

Examples of the memory and the storage devices may include various typesof volatile or non-volatile storage mediums. The examples of the memorymay include a ROM and a RAM.

Therefore, the methods according to the embodiments of the presentdisclosure may be realized in a manner that is performable on thecomputer system. When the methods according to the embodiments of thepresent disclosure are performed on a computer apparatus,computer-readable commands may be executed to perform the methodsaccording to the present disclosure.

It is possible that the above-described above methods according to thepresent disclosure are realized, as computer-readable codes, on acomputer-readable recording medium. Computer-readable recording mediumsinclude all types of recording mediums on which data that can be decodedby the computer system are stored. Examples of these computer-readablerecording mediums include a read only memory (ROM), a random accessmemory (RAM), a magnetic tape, a magnetic disk, a flash memory, anoptical data storage device, and the like. In addition, the methodsaccording to the present disclosure may be stored and performed as codeson the computer-readable recording medium, the codes being distributedto computer systems connected over a computer network and being readablein a distributed manner.

FIG. 104 is a view illustrating a purpose-built vehicle according tostill another embodiment of the present disclosure.

With reference to FIG. 104 , a purpose-built vehicle (PBV) 100 includesa skateboard 110, a cabin 120, and a seat 130.

The skateboard 110 constitutes a bottom surface of a vehicle body.

The cabin 120 is combined with an upper end portion of the skateboard110 and thus provides a riding space and a separate accommodation space.

For example, because a height of a bottom surface of the cabin 120 issmall, in order to secure the field of vision of a driver's seat, aheight of the seat 130 has to be increasingly adjusted to a positionwhere the line of visibility is secured. Accordingly, the separateaccommodation space can be secured in a lower end portion of the seat130 installed in the riding space in the cabin 120.

FIG. 105 is a vertical cross-sectional view taken along line A-A on FIG.104 .

With reference to FIG. 105 , a battery mounting space 111 is formed inthe skateboard 110. The battery mounting space 111 has to be designedconsidering the degree of precision of installation of a battery (notillustrated) and the degree of precision of control thereof.

For convenience of the occupant, the cabin 120 has to be designed insuch a manner that the seat 130 is not only reclinable, but alsorotatable by itself. Particularly, the cabin 120 may be designed in sucha manner that an overall height L thereof is such that the occupant canbe in place on his/her legs.

At this point, the overall height L of the cabin 120 may be adjustedunder separate control.

A display 121 installed in the cabin 120 may transmit an entertainmentscreen image for infotainment and other informational images to ascreen.

A windshield 122 selectively presents an image toward the inside and theoutside of the vehicle.

The windshield 122 includes an electrochromic image region 122_1 and anon-transparent image region 122_2.

Shading of the electrochromic image region 122_1 is adjustable in such amanner that, according to application of electric power, an image isselectively presented toward the inside and the outside of the vehicle.

The non-transparent image region 122_2 extends from a lower end of theelectrochromic image region 122_1.

As another example, the windshield 122 may be divided into an internaldisplay region and an external display region.

For example, the internal display region may present the image towardthe inside of the vehicle, and the external display region may presentthe image toward the outside of the vehicle. At this point, the internaland external display regions may have respective electricity applicationpaths that are separated from each other.

FIGS. 106 and 107 are horizontal cross-sectional views taken along lineA-A on FIG. 104 .

FIG. 106 illustrates an example where the occupant riding in thepurpose-built vehicle 100 watches an image on the display 121 byperforming switching to an autonomous traveling mode and activatesfront-side and rear-side left speakers 123_1 and 123_2 of an internalspeaker 123.

A position of the seat 130 in each row is adjustable according to need.Thus, the occupant may watch the corresponding image in such a mannerthat the position of the seat 130 is comfortably adjusted to a positionfacing the display 121 installed inside of the vehicle.

At this point, the internal speaker 123 may provide sounds through aplurality of speakers 123_1, 123_2, 123_3, and 123_4 that are connectedto each other, under the control of the occupant and/or according to abasic setting.

With reference to FIG. 107 , in a situation in FIG. 106 , the occupantmay separately detach a sitting base of the seat 130 and may watch anexternal image on the windshield 122 after the sitting base is fitted inplace outside of the vehicle.

At this point, the external speaker 124 may provide sounds through aplurality of speakers 124_1 and 124_2 that are connected to each other,under the control of the occupant and/or according to a basic setting.

FIGS. 108 to 114 and FIG. 115 are views illustrating various seatstructures, respectively, of the purpose-built vehicle according to thepresent embodiment of the present disclosure.

With reference to FIGS. 108 to 114 and FIG. 115 , the seat 130 includesa frame 131 and a sitting base 132.

The frame 131 is rotatable by itself and is reclinable in an adjustablemanner. A headrest of the frame 131 is adjustable, and a lumber supportthereof is adjustable.

In addition, the frame 131 is rotatable by itself, is reclinable by apredetermined angle α in an adjustable manner, and is movable backwardand forward by a predetermined distance in an adjustable manner.

The frame 131 operates in conjunction with an ECU (not illustrated) thatis possibly synchronized to big data. The frame 131 transmits a weighton the seating base 132 to the ECU. The ECU performs computation forcomparison of the weight with statistical information accumulated in thebig data. Then, the ECU may adjust a position of the frame 131 byapplying a value of a result of the computation or may enable theoccupant to check the corresponding information using the display and toselect the position of the frame 131.

As another example, the position of the frame 131 is adjustable in amanner that is moved along a rail (not illustrated) installed within thecabin 120.

The seating base 132 is fastened to the frame 131 in a snap-fit way andthus provides a sitting space.

The seating base 132 may be configured to have such a rotatablestructure that the seating base 132 is detached from the frame 131 ofthe seat 130 and then is possibly fitted in place in an arbitrary space.

In this case, the seating base 132 may have a separate grip groove 133in a lower end portion thereof that is brought into contact with theframe 131 in such a manner that seating base 132 is easily detachable toand attachable from the frame 131. The grip groove 133 may be formed inthe shape of a handle.

An upper end surface of the seating base 132 has such a curved structurethat the upper end surface thereof is increasingly inclined upwardtoward the front. A lateral surface of the seating base 132 may beconfigured to have such a curved structure that the lateral surfacethereof is increasingly expanded outward toward a downward direction inorder for the user's calves to rest on the lateral surface thereof.

It is preferable that the seating base 132 is made of carbon fiberreinformed plastic (CFRP) to ensure rigidity for sitting on the seatingbase 132.

The seating base 132 may be configured to have such an integratedstructure that three surfaces thereof are connected to each other.

A plurality of ribs are arranged in each of several sections of theinside of the seating base 132, and thus rigidity of the seating base132 may be further enhanced.

As another example, an inner surface of the seating base 132 may beconfigured to have a honeycomb structure, and thus the rigidity of theseating base 132 may be further enhanced.

The seating base 132 has such a fitting structure that seating on theseating base 132 is possible by rotating supports 134 that are capableof supporting at least two surfaces of the seating base 132,respectively. In this case, it is important that the seating base 132 isfitted in place while keeping in parallel with a floor by settinglengths of the supports 134 and angles of rotation thereof. It ispreferable that the seating base 132 is configured to have a three pointor four point support structure.

The seating base 132 may include a cushioning member (not illustrated),capable of absorbing an impact, on an edge thereof that is brought intocontact with the frame 131. In this case, the cushioning member isfastened to an inner edge of the seating base 132. Thus, the cushioningmember cushions against a weight of the occupant sitting on the seatingbase 132 and absorbs an impact between the seating base 132 and theframe 131.

The seating base 132 in FIG. 113 includes the supports 134, a hingeshaft 135, and fixation end portions 136.

The supports 134 have a structure where they are rotatable with respectto both lateral surfaces, respectively, of the seating base 132.

The hinge shaft 135 rotatably connects the supports 134 to the bothlateral surfaces, respectively, of the seating base 132.

The fixation end portions 136 may be positioned on respective sections,respectively, of lateral end portions of the seating base 132 in such amanner that the supports 134 are kept fitted in place in a state ofbeing rotated outward. That is, the fixation end portions 136 mayfunction as stoppers that fix the supports 134 that serve as stands.

The seating base 132 in FIG. 114 includes the supports 134 and brackets137.

In this case, the supports 134 are connected to the both lateralsurfaces, respectively, of the seating base 132 in such a manner thatthey are rotatable with respect thereto.

The brackets 137 may be fastened to respective sections, respectively,of the lateral end portions of the seating base 132 in a snap-fit way insuch a manner that the supports 134 maintain rigidities thereof in astate of being rotated outward.

FIG. 115 is a view each illustrating a shape of the support 134 whenviewed from behind the support 134.

The supports 134 are not limited to these exemplary shapes.Consequently, the above-mentioned shapes of the supports 134 aresignificant in enhancing the rigidities thereof.

The present disclosure is not limited to the embodiments describedabove. The embodiments described above may be practiced in a modifiedmanner within the scope that does not depart from the technical idea ofthe present disclosure.

What is claimed is:
 1. A system for vehicle display image warpingcomprising: an occupant monitoring unit configured to check occupantmonitoring information; a control command generation unit configured togenerate a control command for warping an image according to the resultof monitoring the occupant; and a warped image display unit configuredto display a warped image according to the control command.
 2. Thesystem of claim 1, wherein: the occupant monitoring unit checks at leastone of a position of the user who rides in the vehicle, information on aseparation distance between a reference point of the vehicle display andthe user, and gazing position information of the occupant.
 3. The systemof claim 1, wherein: the control command generation unit generates thecontrol command for determining the degree of warping according to theresult of monitoring the occupant.
 4. A method for controlling vehicledisplay image warping comprising: (a) monitoring an occupant; (b)generating a control command for warping an image according to a resultof monitoring of the occupant; and (c) displaying a warped imageaccording to the control command.
 5. The method of claim 4, wherein step(a) checks at least one of the riding position of the occupant who ridesin the vehicle, the information on the separation distance of the userto the reference point of the vehicle display, or the gazing positioninformation of the occupant, and step (b) generates the control commandfor determining the degree of the warping according to a result of thechecking in the step (a).
 6. The method of claim 4, wherein step (c)displays the warped image, in which the height of one end of the warpedimage that is close to the occupant is set to be smaller than the heightof the other end of the warped image that is relatively remote from theoccupant.